Hallo,
I have migrated some project from Talend Open Studio 5.6.1 to 6.1.0 - mostly with no problems. But the job generation seems to be very slow.
So I created some very simple jobs, a MainJob and 10 empty subjobs. The MainJob contains 10 tRunJob-Components that call the empty subjobs. Here a comparison of the job-generation:
Talend 5.6.1: < 1s
Talend 6.0.1: ~ 2.5s
Talend 6.1.0: ~ 5s
Then I migrated some "production" jobs to version 6.1.0. This are mostly normal etl-processes to build up dimension-tables for a DWH-database.
Job-Generation:
Talend 5.6.1: ~10s
Talend 6.1.0: ~51s
And this were quite "small" jobs. I have a bigger Job that needs about 4-5 Minutes to export on 5.6.1. After 15 Minutes I killed the export-process on 6.1.0.
Is there anything setting to speedup the job-generation? Any maven settings? Any parallelization for job generation?
Hi, Is there any error message in log which is located in <Talend Studio installation path>/workspace/.metadata/.log when you kill your processing? Does this issue only repro studio 6.1.0? Have you tried to generate one job from one studio to another to see if the speed is normal?
Hi,
no there is no error message. I think, the process would be generated but it takes very long.
If I run one simple job from Talend, there is almost not difference:
5.6.2: < 0.5s
6.0.1: ~1.2s
6.1.0: ~1.3s
This where very simple Jobs in a very small demo project.
My normal project to build up a DWH has about 700 Jobs.
A comparison in Job-Generation if I run the job in studio:
5.6.1: < 0.5s (second run)
6.1.0: ~ 5.5s (second run)
Building the Job as Standalone Job
5.6.1: ~ 6s
6.1.0: ~ 25s
This is for a "simple" job with a MSSQL-Server as source and 5 Lookup tables, two tMaps and a MSSQL-Output.
For me it seems, that there is a high overhead for every job and at the end of the job generation in the new version.
There are no error messages and no missing Jar-Files.
Hi Sabrina,
I'm having the same issue. We have a big job that calls almost 30 Subjobs. It takes me about 30 mins to build that big job. However, it was only about 5 mins when I was using Talend v5.5.
Hi,
are there any possibilities to gain more performance from the Job build?
I noticed that all job are generated over and over again e.g. I have a tRunJob with a dynamic job and I have selected about 200 Job als possible Jobs. It I generate the start-Job, all 200 Jobs are generated every time, even if nothing has changed in the jobs.
This is not very efficient. Are there any hidden settings for that?
Kind regards,
Paul Mandalka
I am having the exact same issue. This needs resolving. I have raised a support call.
Talend Studio should be able to identify which jobs have been modified and ONLY compile those (obviously jobs containing jobs which have been modified will need to be recompiled as well). We have a situation where a job that took 5 to 10 mins to compile in 5.6.1 takes almost an hour in 6.1.1.
Hi,
We have recently migrated to Talend 6.1.1(Enterprise version) and we are facing the same issue.
Please let me know if there is any solution for this.
Thanks.
