merge multiple xml files with same schema into one through talend

Anonymous — Thu, 17 Mar 2016 19:37:37 GMT

I have a talend job which runs for huge dataset and create an xml at the end .In order to avoid memory issue ,i split the xml into batches of 100 each ,depends on number of books in the dataset.
lets say i have 300 books,so after processing of job ,3 files are created 1.xml,2.xml and 3.xml all contains data with same schema. Now i need to combine the xml into one xml which seems to be difficult to implement in talend for complex schema's.
My job looks like

 tfileList--iterate-->tFileInputXml--merge-->tUnite---main--->tAdvancedFileOutputXml

For simple schema it works fine as show below

<?xml version="1.0"?>[/font][/size][/color]
<x:books xmlns:x="urn:books">[/font][/size][/color]
   <book id="bk005">[/font][/size][/color]
      <author>Writer</author>[/font][/size][/color]
      <title>The First Book</title>[/font][/size][/color]
      <genre>Fiction</genre>[/font][/size][/color]
      <price>44.95</price>[/font][/size][/color]
      <pub_date>2000-10-01</pub_date>[/font][/size][/color]
      <review>An amazing story of nothing.</review>[/font][/size][/color]


   </book>[/font][/size][/color]


   <book id="bk006">[/font][/size][/color]
      <author>Poet</author>[/font][/size][/color]
      <title>The Poet's First Poem</title>[/font][/size][/color]
      <genre>Poem</genre>[/font][/size][/color]
      <price>24.95</price>[/font][/size][/color]
      <review>Least poetic poems.</review>[/font][/size][/color]
   </book>[/font][/size][/color]
</x:books>[/font][/size][/color]




If i change the xml as shown below ,it will be difficult to think because we can have only one loop element

<?xml version="1.0"?>[/font][/size][/color]
<x:books xmlns:x="urn:books">[/font][/size][/color]
   <book id="bk001">[/font][/size][/color]
      <author>Writer</author>[/font][/size][/color]
      <title>The First Book</title>[/font][/size][/color]
      <genre>Fiction</genre>[/font][/size][/color]
      <price>44.95</price>[/font][/size][/color]
      <pub_date>2000-10-01</pub_date>[/font][/size][/color]
      <review>An amazing story of nothing.</review>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>10</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>20</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>10</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
   </book>[/font][/size][/color]


   <book id="bk002">[/font][/size][/color]
      <author>Poet</author>[/font][/size][/color]
      <title>The Poet's First Poem</title>[/font][/size][/color]
      <genre>Poem</genre>[/font][/size][/color]
      <price>24.95</price>[/font][/size][/color]
      <review>Least poetic poems.</review>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>210</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>310</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
      <distributions>[/font][/size][/color]
         <total_value>410</total_value>[/font][/size][/color]
      </distributions>[/font][/size][/color]
   </book>[/font][/size][/color]
</x:books>[/font][/size][/color]

The xml is more complex ,but i need to implement this ,please help me with this

Re: merge multiple xml files with same schema into one through talend

Anonymous — Mon, 21 Mar 2016 03:26:18 GMT

Hi,
tAdvancedFileOutputXML don't support multiple loop element?
Please take a look at component?TalendHelpCenter:tXMLMap which is an advanced component fine-tuned for transforming and routing XML data flow (data of the Document type), especially when processing numerous XML data sources, with or without flat data to be joined.
Best regards
[font=noto, Helvetica, Arial, sans-serif]Sabrina[/font]

topic merge multiple xml files with same schema into one through talend in Talend Studio

merge multiple xml files with same schema into one through talend

Re: merge multiple xml files with same schema into one through talend