topic Reading only xml file from un-archived folder through tfilelist in Talend Studio

Reading only xml file from un-archived folder through tfilelist

Anonymous — Sat, 16 Nov 2024 11:08:41 GMT

Hi All ,
I am reading zip files from a folder unzipping that using tFileUnarchive. After Unzipping , i am reading only xml files using tfilelist after tFileUnarchive. I am iterating 2 files in parallel. But when i do so my whole process runs as many times as many are the files.
My workflow looks like this:
tfilelist --> tfileunarchive --> tfilelist --> tfileinputxml --> tlogrow
i have 2 zip folders one contains 2 files (1 xml and 1jpeg) and other zip contains ( 1 xml ). I only want to process xml files after unzipping. When i iterate it with number of parallel execution as 2 it runs the workflow twice for 3 files.

Output that i am getting through this workflow is :
Process finished
273985234|ZO|1237351|JM|TestData|99
273985234|ZO|1237351|JM|TestData|99
273985234|ZO|1237351|JM|TestData|99
273985234|XO|1237161|SM|TestData|34
273985234|XO|1237161|SM|TestData|34
273985234|XO|1237161|SM|TestData|34

Expected Output:
273985234|ZO|1237351|JM|TestData|99
273985234|XO|1237161|SM|TestData|34

Please find attached screenshot of the workflow.

Thanks,
Saurabh.

Re: Reading only xml file from un-archived folder through tfilelist

Anonymous — Tue, 14 Jul 2015 07:54:43 GMT

Hi,
A parallelization-enabled Iterate connection allows the component that receives threads from the connection to read those threads in parallel.
Have you set any file mask in tFileList_2?(*.xml)?

Best regards
Sabrina

Re: Reading only xml file from un-archived folder through tfilelist

Anonymous — Tue, 14 Jul 2015 08:32:09 GMT

Yes i have set the mask as "*.xml"

Re: Reading only xml file from un-archived folder through tfilelist

Anonymous — Tue, 14 Jul 2015 09:10:24 GMT

Hi,

A parallelization-enabled Iterate connection allows the component that receives threads from the connection to read those threads in parallel.

Please disable option "Enable parallel exection" in iterate row to see if it works.
Best regards
Sabrina

Re: Reading only xml file from un-archived folder through tfilelist

Anonymous — Tue, 14 Jul 2015 17:01:01 GMT

No it still process the files twice.Only when i remove tfilelist_2 it process it once. But in that case it process even the image files that are their in unzipped folder and gives me error for those.

Re: Reading only xml file from un-archived folder through tfilelist

Anonymous — Wed, 15 Jul 2015 06:57:18 GMT

It only works when i introduce delete component , but even for that it works only with "Enable parallel exection" =1
I am attaching the config details of tfilelist_2 and tfiledelete_1. Output that i am getting using the specified configurations is:

[font=Verdana, Helvetica, Arial, sans-serif] Starting job prcs_zip_chronological_order at 11:23 15/07/2015.[/font]

[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] connecting to socket on port 3941[/size][/font]
[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] connected[/size][/font]
[font=Verdana, Helvetica, Arial, sans-serif] Processing archive E:\Blogtestdata\abctest.zip, please wait...[/font]

[font=Verdana, Helvetica, Arial, sans-serif] Process finished[/font]
[font=Verdana, Helvetica, Arial, sans-serif] 273985234|XO|1237161|SM|TestData|34[/font]
[font=Verdana, Helvetica, Arial, sans-serif] Processing archive E:\Blogtestdata\cdftest.zip, please wait...[/font]

[font=Verdana, Helvetica, Arial, sans-serif] Process finished[/font]
[font=Verdana, Helvetica, Arial, sans-serif] 273985234|ZO|1237351|JM|TestData|99[/font]
[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] disconnected[/size][/font]

[font=Verdana, Helvetica, Arial, sans-serif][size=1]Job prcs_zip_chronological_order ended at 11:23 15/07/2015. [exit code=0][/size][/font]
Thanks,
Saurabh.