topic Returning multiple rows from subjob in Talend Studio

Returning multiple rows from subjob

Anonymous — Mon, 28 Sep 2015 19:30:16 GMT

Folks,
I would think this would be basic/simple but I cannot get it to work. I am trying to logically break up my workflow into a series of subjobs that can be reused easily. My overall workflow is to retrieve 2 files (each one from a different MQ) then parse each file as fixed width delimited and then finally join the two files together into one result dataset based on a key common to the content of both files.
I have broken out the parsing of a file to a child job - in there it contains tFileInputPositional - if I use a tLogRow component within the subjob I correctly see each row being output to the logs. However, if I pass the tFileInputPositional directly to a tBufferOutput for return to the parent job, nothing ever gets returned - I.e. back in the parent if I put a tBufferOutput from the output of the subjob, I get no rows back.
Is tBufferOutput designed to handle multple rows? If so, can someone give me an example of this at work. If I am using the wrong component, what component should I use?

Re: Returning multiple rows from subjob

Anonymous — Tue, 29 Sep 2015 01:27:23 GMT

Yes, tBufferOutput is used to returns data back to parent job, you need to define the same schema as tBufferOutput's schema on tRunJob which is calling the child job. eg:
child job
tFileInputPositional--tBufferOutput
parent job;
tRunJob--main--tLogRow
In this example, you will see the result are printed on the console.
Best regards
Shong

Re: Returning multiple rows from subjob

Anonymous — Wed, 30 Sep 2015 14:44:51 GMT

Shong,
Thanks for replying and that IS how I have it set up. However, I've solved my pesky problem: Under 'advanced settings' for the tRunJob that represents the child job in the parent, I had to ensure 'propagate the child result to the output schema' for the results to return. You'd think that *duh* I would have known this but the thing is, there are several times I have used sub-jobs (such as processing data off MQ, etc.) and I have NEVER had to check this before to get my data back to the parent. Which makes this a confusing inconsistency for me if anyone is able to explain?
thanks,
D.