issue with parallel execution of jobs through tparallelize

sushantk19 — Fri, 15 Nov 2024 22:56:16 GMT

Hello friends,

I have the below scenario:

we are running 2 talend jobs parallely in talend management console. Both are scheduled to run every night.

but from past 2 days, i observed only one of the job is executing correctly and loading the data to the target table, while count of rows for the other job is 0 causing my subsequent dependent jobs to fail. It was all working fine from past 1.5 years, but since 2 days its failing to load the data.

What could be cause for this? is it some issue with memory as the volume of data keeps increasing.

when i checked the parallelize buffer size in talend i see that default value is set to 25000. Can we increase this value? how much should be keep.

From my IT team i found that we have 8 gb memory associated with our remote engine. volume of the data in 2 jobs/tables is roughly 8000 rows per day.

Any pointers would be appreciated :). I have attached the screenshot of the parallelize buffer size in talend for your reference.

Re: issue with parallel execution of jobs through tparallelize

hp1 — Wed, 11 May 2022 16:41:56 GMT

What do the logs display for the jobs invoked through the tRunJob? Information from the logs will help understanding more on what is happening.

topic Re: issue with parallel execution of jobs through tparallelize in Talend Studio

issue with parallel execution of jobs through tparallelize

Re: issue with parallel execution of jobs through tparallelize