Passing Component Reference

Anonymous — Sat, 16 Nov 2024 06:49:28 GMT

Hi all,

Not sure if anyone faced this issue with parent and child job execution.

In my flow, the parent job loads data from multiple CSV files, processes it and feeds the child job which reads a matching document from MongoDB and updates it or inserts new one, if nothing is found.

As part of the execution, all the data is prepared by the parent job. Since child job establishes connection to MongoDB, reads and writes to Mongo and closes the connection, my test execution is very time consuming. As some point, I had to slow down the parent job feeding the child job because of not able to get enough MongoDB connections in the child job, which has made the situation bad. When I moved all the components from child job to the parent job by eliminating parent/child, the test execution flies. To put in numbers, its about 200% improvement in time for processing the same volume of data.

I want to know if there is a way to pass MongoDBConnection reference from parent job to child job so the child job doesn't have to deal with connection management.

I am using TOS 6.4.0 on Windows.

Thanks,

SRK.

Re: Passing Component Reference

Anonymous — Mon, 21 Jan 2019 07:21:47 GMT

Hello,

Could you please try tContextLoad to load the context values to the context variables and configure the input connection with the context variable?

Here is a related scenario about:TalendHelpCenter: Context scenario

Best regards

Sabrina

topic Re: Passing Component Reference in Talend Studio

Passing Component Reference

Re: Passing Component Reference