I am trying to accomplish the following: load data from a database once, replicate the data flow, then pass each flow to a different child job to process the data. I have gotten the first two steps to work using the tMysqlInput and tReplicate components. However, I am having trouble with the third step. I tried connecting the data flow to a tRunJob component, but it seems that the tRunJob component is not designed to take data input. Is this correct? How would you pass a data flow to a child job?
I just searched for the answer to this same question. I believe if you add a tBufferInput to your subjob, you should be able to attach a dataflow to the job. I haven't yet tried to run the job myself, but I'm interested to see what other responses there are.
You need define some context vars in child job and receive input flow on tRunJob, your job looks like:
father job:
tMysqlInput---tReplicate--tFlowToIterate---iterate--tRunJob
child job:
tFixedFlowInput---other components
tFixedFlowInput: generated a fixed flow
About how to pass a variable from father job to child job, see 1654.
Best regards
shong
Shong,
Thanks for your quick reply. I think I understand what your are suggesting and have seen discussion of this elsewhere on the forum. Is it true then that you cannot pass multiple rows of data into a child job at one time? Can you explain why it is not possible to simply connect a flow of data to a tRunJob and have it passed to the child job as with any other data flow component?
Thanks!
I just searched for the answer to this same question. I believe if you add a tBufferInput to your subjob, you should be able to attach a dataflow to the job. I haven't yet tried to run the job myself, but I'm interested to see what other responses there are.
Andrew, I tried using tBufferInput and tBufferOutput to pass data into a child job, but it did not work. I have seen examples of how you can use them to pass data out of a child job using them, but it doesn't sound like it works for passing data in.
Can you explain why it is not possible to simply connect a flow of data to a tRunJob and have it passed to the child job as with any other data flow component?
Good question!! In fact, we do that in our commerical version Talend Integration Suite with Joblet feature.
Best regards
shong
I am also having same problem not able to pass data flow to tRunjob. i tried using joblet which worked fine in passing dataflow. I need jar file of each subjob, which is possible in case of using job which creates jar file for each subjob. But when I use joblet no jar file is created for joblets. I m stuck in middle. I need to pass flow to subjob and also want jar file for subjob. subjob and joblet only fullfill one requirement. Is there any way by which my both requirements are met.
Andrew, I tried using tBufferInput and tBufferOutput to pass data into a child job, but it did not work. I have seen examples of how you can use them to pass data out of a child job using them, but it doesn't sound like it works for passing data in.
unfortunatly you're right.
in TOS you have to use some temporary file or other 'trick' to propagate data from father to child.
regards
laurent
