topic Re: XML file loop in Talend Studio

XML file loop

LDPtechAFB — Sat, 16 Nov 2024 02:00:32 GMT

Hello,

i'm beginning in Talend Studio, and i want to implement database with all the informations of my XML file.

I tried with some component (tXMLmap, tmap, tWriteXMLfield, tlogrow), and i sent all datas in tfileoutputraw component (to verify that xml treatment work well).

But the issue is that have only one iteration of each tag. I tried to change the XPath, put a loop but i already have this issue.

cf: forum1.png show you the tree view of my XML file.

the issue is that element in my machines nodes have average 1000,1500 iterations and only one is send to the rawfile.

Hope you will understand my request.

Regards

Re: XML file loop

Anonymous — Wed, 01 Jul 2020 11:11:21 GMT

I'm not entirely sure that I understand your issue I'm afraid. I *think* it might be related to this one that I provided a solution for. Can you take a look and let me know....

https://community.talend.com/t5/Design-and-Development/Complex-XML-with-multiple-loops-into-one-output/td-p/36488

Re: XML file loop

LDPtechAFB — Fri, 03 Jul 2020 15:26:36 GMT

Yes it helped me a lot, to understand how it work !

I tried many things , but once datas go through the tExtractXMLField component, iterations are "deleted" and When i look (thanks to a Tlogrow component) before the component, all the datas are Ok, but when i look after, i only get one iteration.

The format of my datas looks like

...

</StemType>

</SpeciesGroupDefinition>

...

</DiameterClasses>

</ProductDefinition>

</Machine>

Tag : <SpeciesGroupDefinition> , <StemType> , <ProductDefinition> , <DiameterClasses> are repeating in the file.

And I need all the informations if these tag.

So I tried to Implement solution with flows replication and then using XMLExtract (forum2.PNG).

Then I configure my source tinputXML file like the forum1.PNG (HarvestProduction is the parent node of Machine).

and I replicate the flow, configure my extract component like forum3.PNG.

But I get the result of forum4.PNG (Only one result instead of 7,8 iteration)

Hope you understand my problem Better

regards

forum5.PNG
forum3.PNG
forum2.PNG
forum1.png
forum4.PNG

Re: XML file loop

LDPtechAFB — Fri, 03 Jul 2020 15:26:36 GMT

Yes it helped me a lot, to understand how it work !

The format of my datas looks like

...

</StemType>

</SpeciesGroupDefinition>

...

</DiameterClasses>

</ProductDefinition>

</Machine>

Tag : <SpeciesGroupDefinition> , <StemType> , <ProductDefinition> , <DiameterClasses> are repeating in the file.

And I need all the informations if these tag.

So I tried to Implement solution with flows replication and then using XMLExtract (forum2.PNG).

Then I configure my source tinputXML file like the forum1.PNG (HarvestProduction is the parent node of Machine).

and I replicate the flow, configure my extract component like forum3.PNG.

But I get the result of forum4.PNG (Only one result instead of 7,8 iteration)

Hope you understand my problem Better

regards

Re: XML file loop

Anonymous — Tue, 07 Jul 2020 16:47:22 GMT

I think I see your issue. I am afraid I am a bit snowed under with some work that you guys should hopefully see the result of next week, but I may be able to give an example of how to extract this data if you can give me an example file with dummy data. Just make sure that the file has examples of all of the loops and this should do. Remember, make sure that it is dummy data

Re: XML file loop

Anonymous — Tue, 07 Jul 2020 16:47:22 GMT

Re: XML file loop

LDPtechAFB — Wed, 08 Jul 2020 14:34:14 GMT

Hello,
Here is the file with dummy datas.

Thanks for your involvement.

I didn't find a solution yet.

i'm trying a lot, of things, consulting multiples post/forums but i think the issue is my lack of knowledge. Maybe i did something wrong.

Thank you for your answers

JDtest.xml

Re: XML file loop

LDPtechAFB — Wed, 08 Jul 2020 14:34:14 GMT

Hello,
Here is the file with dummy datas.

Thanks for your involvement.

I didn't find a solution yet.

i'm trying a lot, of things, consulting multiples post/forums but i think the issue is my lack of knowledge. Maybe i did something wrong.

Thank you for your answers

Re: XML file loop

Anonymous — Wed, 08 Jul 2020 19:43:46 GMT

I have put together a quick job that I think covers what you want. Hopefully my screenshots will tell the story, but I will add some descriptions where I think more info is needed.

The above is a screenshot of the job. It is split into 3 paths; one for the Machine data, one for the SpeciesGroupDefinition data and one for the ProductDefinition data. This split takes place at the tMap_1.

The tFileInputXML_l just receives the XML and outputs an XML Document to the next component. This component is configured as below...

To see the whole post, download it here
OriginalPost.pdf

Re: XML file loop

LDPtechAFB — Wed, 15 Jul 2020 07:32:20 GMT

hello @Richard Hall

, I saw your answer last week (but we cannot see it anymore, i have your answer in my mailbox but without the attached pictures), It helped me a lot.

The problem is that datas concatenate to other each time a TExtractRow send datas to another one (it create me a 0.5Gb file, while the original file is 0.005gb).

i tried to to extract datas One by one (node by node) to extract and send thanks to a TmapComponent and merge them, but the resultat is the same.

(1024 rows)

don't want to have this amount of duplication

How can i associate the good iterations together ?