Folks, I have a sub job that takes input, two flat files, parses them out and joins them based on a common key - the final resultset is returned back to the parent job. I now want to pass this dataset to another subjob to handle some data conversion. I'm trying to find a good example of how to pass a multi-row dataset to a sub job. I've searched the forums and the help guides with little success. Can anyone either point me in the right direction or give me an example of how it is done? Currently I am using a tFlowToIterate to pass one row at a time from the parent to the sub job but I find this mechanism less than ideal. thanks!
Hi dpower, this can done using a bit of Java as in this tutorial (
http://www.rilhia.com/tutorials/talend-connect-example - this is actually for something else, but uses a method to pass datasets to subjobs that works for me). You can also do it by using this component (
tDataRowToContext - Found in Talend Exchange) that I put together for this very purpose.
Passing large datasets to child jobs can use a lot of memory, so beware of heap space issues.
