TAC - Running parallel job using tParallelize

Anonymous — Sat, 22 Dec 2018 15:27:36 GMT

Hi,

I have designed a job to use tParallel component which has say 4 tRunJob's, where each tRunJob call the same child job.

The child job connects to a database, read the table, processed the rows using tMap, tDataShuffle and tDataMasking components, updates the same table and closes the connection.

This is to split the table to 8 splits and process the rows in parallel. The split is done based on the primary key of the table.

tParallel ----> tRunJob_1(job_process_table1 - say for 100 rows)

----> tRunJob_2(job_process_table1 - say for 101 to 200 rows)

----> tRunJob_3(job_process_table1 - say for 201 to 300 rows)

----> tRunJob_4(job_process_table1 - say for 301 to 400 rows)

This job runs fine when run locally from Talend to the dev databases.

But fails with "Cannot establish connection error" when running from TAC.

The same job runs fine when I run with single tRunJob_1.

Could not understand the reason.

Could you kindly suggest what all might be the reason.

Thanks.

Re: TAC - Running parallel job using tParallelize

Anonymous — Mon, 24 Dec 2018 04:52:41 GMT

Hi,

Are you trying to use the same database connection in all the child jobs?

Since you are closing the connection within the child job, this could be creating some issue. Could you please try separate DB connections for your child jobs in this case and try to see whether the issue persists?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

topic Re: TAC - Running parallel job using tParallelize in Talend Studio

TAC - Running parallel job using tParallelize

Re: TAC - Running parallel job using tParallelize