Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Soumya_M
Contributor II
Contributor II

Loading of large XML files to Postgres

Hello, I have created a talend job which extracts xml file and loads it to postgres. But I can only load files having size in kb, when i tried loading xml file (size 53mb), it is not appearing under File XML option. I need help with this.

Labels (4)
10 Replies
Anonymous
Not applicable

Hello,

Did you attempt to load the large xml through the File XML wizard?

Could you please upload some screenshots of job? So that we can see which components you are using in the job and the detailed settings, and then will give you some suggestions.

Best regards

Sabrina

Soumya_M
Contributor II
Contributor II
Author

I'm unable to fetch the xml file or xml metadata under File XML wizard. Though it gives the schema preview successfully, the xml file does not appear under File XML wizard.

jlolling
Creator III
Creator III

Of course not. I would shorten the file for the purpose of getting the schema a lot. Then studio will certainly internal die because of memory problems.

Soumya_M
Contributor II
Contributor II
Author

How to shorten the xml file ?

Soumya_M
Contributor II
Contributor II
Author

I'm unable to fetch the xml file or xml metadata under File XML wizard. Though it gives the schema preview successfully, the xml file does not appear under File XML wizard.

Anonymous
Not applicable

Hello,

Here is an approach that we used it before. It is reading the big XML as CSV file with tFileInputDelimited componenet and then passing data to tFileOutputXML or tAdvancedFileOutputXML which split it into small XML files.

Best regards

Sabrina

Soumya_M
Contributor II
Contributor II
Author

May i know how to split the resulting tFileOutputXML into small XML files

Anonymous
Not applicable

Hello,

Here is an option "Split Output in several files" in Advanced settings of tFileOutputXML component.

If the output is big, you can split the output into several files, each containing the specified number of rows.

0695b00000bDw9jAAC.pngBest regards

Sabrina

 

Soumya_M
Contributor II
Contributor II
Author

I shall try it, thank you 🙂