topic Re: parallel execution in iteration link : jumbles variable values in Talend Studio

parallel execution in iteration link : jumbles variable values

jmd — Thu, 14 Mar 2013 16:26:45 GMT

Hi,
I tried to use the parallel execution property in iteration link, to run commands (in tSystem component) in parallel
Several commands are launched simultaneously but the parameters passed to them (via context variables and a command line) are mixed
the general schema is the following :
tMySQLInput => tFlowtoIterate =iteration link=> tJava (to treat values returned from tMySQLInput) => tSystem
For example :
the tMySQLInput returns 2 identifiers which are passed to the tSystem component
there are indeed 2 iterations but twice the same identifier is passed to the tSystem component
Have you ever noted this ?
Is there another way to do it ?

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Fri, 15 Mar 2013 09:50:44 GMT

Hi
It sounds like a bug, which version are you using? To me, each iteration should process a different row. Can you please do a testing like this, print the value returned from tMysqlOuput on tJava, for exmaple:
tMySQLInput =row1==> tFlowtoIterate =iteration link=> tJava
on tJava:
System.out.println(row1.identifiers)
Shong

Re: parallel execution in iteration link : jumbles variable values

jmd — Sat, 16 Mar 2013 09:34:33 GMT

Yes, I think it's a bug
I slightly modified the job to use directly the tFlowToIterate variables as you mentionned, instead of context variables (which were set in a tJava component) and it seems to work
context variables seem to be not set in the good moment in the job flow

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Mon, 18 Mar 2013 04:15:34 GMT

Hi
Maybe there is something wrong in your code, how do you set values to context variables? Can you show us your code on tJava? Also, A screenshot of job will help us...
Shong

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Mon, 18 Mar 2013 06:23:42 GMT

HI
Let me add a simple job to your point to judge whether its i a bug or not. I have designed a simple job like
tfileinputdelimited -- main --> tFlowToInterate --iterate(x2) --> tJava --iterate(x2)-->tSystem
I have a collection of commands in tFileInputDelimited, when i running the job the output is confusing me, like it is running the same command multiple times..but without parallel execution in iteration link it is working file. Please find
the Screen shot
thanks
B. Anil Kumar

Re: parallel execution in iteration link : jumbles variable values

jmd — Mon, 18 Mar 2013 12:31:46 GMT

Exactly the same problem in my case : some times , the same command is executed 2 or 3 times and some commands in the list are not executed

Re: parallel execution in iteration link : jumbles variable values

jmd — Wed, 20 Mar 2013 12:59:07 GMT

context.code_echange = input_row.code_echange;
context.groupe_echange = input_row.groupe_echange;
context.id_execution = input_row.id_execution;
context.chemin_fichier_entree = input_row.chemin_fichier_entree;
context.adresse_mail_destinataire = input_row.adresse_mail_destinataire;
context.chemin_sortie = context.EchangesRacine + context.code_echange + "/sortie/";
context.type_envoi = input_row.type_envoi;
https://docs.google.com/file/d/0B15XMMEHVcFlSzE4MTBPd2d5RDA/edit?usp=sharing

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Wed, 20 Mar 2013 13:02:58 GMT

Hi All
Thanks for your information, I will do a testing for this issue and come back to you, keep discussing.
Shong

Re: parallel execution in iteration link : jumbles variable values

jmd — Mon, 25 Mar 2013 11:59:16 GMT

Hi,
any news ?

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Mon, 16 Feb 2015 13:46:15 GMT

Hi Shong,
Any Update on the above discussion?
Thanks---->D

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Tue, 04 Aug 2015 15:28:14 GMT

Hi,
Any update ?

Re: parallel execution in iteration link : jumbles variable values

Anonymous — Tue, 04 Aug 2015 16:10:08 GMT

hi all,
a long time since starting post
never mind ,
first be sure that you have a core for each multi-threading (or it'll be just a like-muli iteration).
second in advanced setting => extra of the job be aware to check 'multi-threading execution'
hope it hepls
regards
laurent