tParallelize - executing multiple jobs in parallel

[Anonymous]

We have designed a job which internally calls multple jobs to load files with certain layout to certain tables. In order to optimize the performance of our jobs we decided to use tParallelize component to run all these jobs in parallel. We made sure that "Multithread option" was checked for the same reason.
When job executed without any issues / errors - we decided to perform QA on populated tables in SQL Server. We found that column values have shifted or swapped for more than one table. There was a case where jobs skipped few rows from source file.
We then executed same job with different approach - run all jobs in sequence instead of in parallel. This time all jobs loaded data correctly for all files.
Screen shot for both the jobs are attached.
We believe this is bug is the component / TIS version we are using. We need to confirm this before putting any bug on jira. This issues is very critical for our project deliverables...

[Anonymous]

Hi
Which version are you using? tParallelize allows you to run different subjobs parallel, this component is only available in Talend enterprise subscription version, if you use tParallelize to run the subjobs parallel, don't need to check the "Multithread option", go to uncheck this option and run the job again, and see if you still get problems.
Shong

[Anonymous]

Hi Shong,
I've tried all the possible combination including the one you've suggested. I'm using 5.2 Enterprise Edition. One more thing I observed while doing this for n number of times - if few column names of different files we are loading using different jobs are same; this issue occurs more frequently. Hope this gives some more clarity on this issue.
Thanks,
Nishad Joshi.

[Anonymous]

Hi Nishad
Thanks for your further explanation, but I will suggest you to report a ticket on our support portal, our colleagues from support team will help you further, we need to investigate if it was a bug or not, maybe a webex is needed to investigate the problem on live.
Shong

[Anonymous]

Alright Shong,
Thank you very much for prompt reply - I'll submit a ticket on support portal for this. Will update this topic once I hear back from support team.
Thanks,
Nishad Joshi.

[Anonymous]

ok, I submitted a ticket on support portal - I got reply that this is design issue, no further explanation. We tried to reproduce this scenario for different data set and it gave us similar results... so the issue is there with this component. Therefore went ahead and submitted a bug on jira -
https://jira.talendforge.org/browse/TDI-26239?page=com.atlassian.streams.streams-jira-plugin%3Aactiv...
If anyone else can work on this and re-produce this again that will be a great help.

[Anonymous]

Hi team , 

tPartitioner, tCollector, tRecollector and tDepartitioner & tParallelize component I didn't find in open source version 6.5.1 .Can  you please help me instead of above motioned component which components can I use if it is not available in open source .

[Anonymous]

I am having same doubt.
Could someone please help regarding above query that which components we can use in Talend Open Source instead of tPartitioner, tCollector, tRecollector and tDepartitioner & tParallelize component as these components are not present in TOS.

[manodwhb]

@ShwetaMishra243 , we do not have corresponding components for tPartitioner, tCollector, tRecollector and tDepartitioner in open studio.

for parallel you can enable Multi thread execution in JOb tab>> Extra .