topic Re: slow job scheduled windows task in Talend Studio

slow job scheduled windows task

gcurato82 — Fri, 15 Nov 2024 23:13:34 GMT

hello everyone, I use Talend open studio 7.3.1 and

I have a problem with my work when it is scheduled with a windows task, because this is slow and takes a long time to run, while if I run it directly from Talend, the work is fast.

the ETL provides a tDBInput object -> tDBOutput, (database postgres) in the input the query retrieves 190 fields and there are 2 joins, are moved less than 400k records.

Does anyone have suggestions on how to fix my problem?

Re: slow job scheduled windows task

Anonymous — Thu, 10 Feb 2022 18:39:59 GMT

This is strange. I would expect the precompiled job to run A LOT faster (as it doesn't have to go through the process of compilation first). Two reasons I can think of that might impact performance (off the top of my head) are....

You have scheduled the job to run during particularly busy times for your DBs
You are running it on a different machine with different resources or different network capabilities

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 08:35:11 GMT

thanks for giving me an answer. the server on which I ran the job, in the two modes, is the same only at different times, at a distance of a few minutes between them. As for the database load, there was no stress at that time. this Job is also scheduled during the night time slot (03:00 AM), and the result is always the same, it takes about 3h.

Re: slow job scheduled windows task

gjeremy1617088143 — Fri, 11 Feb 2022 10:06:53 GMT

Hi, maybe you can improve the xms and xmx param in the .bat of the job.

also when you execute the .bat manualy do you have the same time ?

Send me love and kudos.

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 10:22:39 GMT

hello, thanks for your reply. i had never tried to manually run the job using the file ". bat", I just made the attempt and the execution happened at the same time as when I launch it from Talend console. what can this mean?

Re: slow job scheduled windows task

gjeremy1617088143 — Fri, 11 Feb 2022 10:47:05 GMT

and your scheduled windows task execute this .bat ?

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 10:51:01 GMT

you are executed by the windows task without errors, but very slow.

The task runs only when previous tasks are successfully executed.

Re: slow job scheduled windows task

gjeremy1617088143 — Fri, 11 Feb 2022 10:52:37 GMT

could you try to make a new scheduled task with this job , with no previous task needed to run , you schedule it in the next 5 minute and you'll see if the problem come from the job

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 11:04:35 GMT

hello, I did the test you suggested and the job took a few minutes, as for the Talend console.

so this means that the problem is the trigger with the other tasks?

Re: slow job scheduled windows task

Anonymous — Fri, 11 Feb 2022 11:09:16 GMT

Following on from what @guenneguez jeremy said, maybe try adding some logging to the job (if you haven't already). Just something at the very start (using a tPreJob component) and something at the very end (using a tPostJob) component. From what you have both discussed, this looks like it might not be the job you suspect is slow causing the problems here.

Re: slow job scheduled windows task

Anonymous — Fri, 11 Feb 2022 11:13:56 GMT

It looks like I was writing my response as you were writing yours @Giuseppe Curato. Can you add logging to all of your jobs that are running in this sequence. As suggested in my previous post. Just a start and end time, maybe to a DB table somewhere. This is actually a good idea when running jobs without using our subscription product (where this sort of thing is built in). From this you will be able to see precisely which job is causing this.

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 11:28:34 GMT

the log tracking is already active, there are no overlapping tasks, but if I run the task that is in trigger takes 3h, what I run now with precise time took a few minutes. I don’t understand the difference between the two tasks

Re: slow job scheduled windows task

Anonymous — Fri, 11 Feb 2022 11:58:46 GMT

What is the task before this runs doing? Have you tried to run this task on its own (with nothing else running), using the scheduler?

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 12:19:22 GMT

Yes, I have already tried to run the task manually and the result is always 3h. creating a new task scheduled to a precise time, it took a few minutes. in both cases, the server was unloaded

Re: slow job scheduled windows task

Anonymous — Fri, 11 Feb 2022 13:44:21 GMT

This is getting a bit confusing to follow. Can you confirm or correct the following statements.....

The job run through Talend Studio is quick
The job run through the scheduling tool (with other jobs being triggered or business as usual) takes 3 hours
The job run via the .bat file on the command line takes 3 hours
The job run from the scheduling tool on its own with no other jobs takes 3 hours

Once we have an understanding of the above, we can look at potential causes. I don't believe this is a Talend issue though.

Re: slow job scheduled windows task

gcurato82 — Fri, 11 Feb 2022 19:58:38 GMT

Hi everyone, after doing several tests with tasks and jobs, I came to the conclusion that maybe the problem was related to the previous job that engages the database, so I solved by inserting a timer at the beginning of the job in trigger to postpone the start. I have so solved.