Hallo,
I have migrated some project from Talend Open Studio 5.6.1 to 6.1.0 - mostly with no problems. But the job generation seems to be very slow.
So I created some very simple jobs, a MainJob and 10 empty subjobs. The MainJob contains 10 tRunJob-Components that call the empty subjobs. Here a comparison of the job-generation:
Talend 5.6.1: < 1s
Talend 6.0.1: ~ 2.5s
Talend 6.1.0: ~ 5s
Then I migrated some "production" jobs to version 6.1.0. This are mostly normal etl-processes to build up dimension-tables for a DWH-database.
Job-Generation:
Talend 5.6.1: ~10s
Talend 6.1.0: ~51s
And this were quite "small" jobs. I have a bigger Job that needs about 4-5 Minutes to export on 5.6.1. After 15 Minutes I killed the export-process on 6.1.0.
Is there anything setting to speedup the job-generation? Any maven settings? Any parallelization for job generation?
Hi,
We too are facing same issue with jobs taking too much time to generate & run the jobs.
Does any one have got solution for this issue.
Talend 6.1.1 (Enterprise Edition)
Thanks & Regards
Ashok Karnam
We had the same issue when we upgraded from Talend 5.6 to 6.1. The generate/build step started taking double the amount of 5.6.
Here are my observations and steps followed to bring down to normal.
1. Talend 5.6 requires Java 7 jdk 1.7 (Must)
2. Though Talend 6.1 still support jdk 1.7 but it is highly recommended to have Java 8 jdk 1.8. This is big! give this a try.
3. Talend 6.1 generate/build process includes maven, this is another reason the build process takes more time. This was optional in 5.6 but it is part of build in 6.1.
4. Talend 6.1 requies patch 1237 to address slowness in setComponentPath (a step in generate). If you are generating via TAC, goto commandline section and notice the time it took for initRemote, logonProject, setComponentPath and buildJob.
4.1 If you see delay in initRemote - this is probably connection to TAC is very slow
4.2 if it's logonProject - you may require svn cleanup and delete unwanted jobs in repository
4.3 if it's setComponentPath - the patch 1237 will fix it.
4.4 If it's buildJob step - java upgrade will hopefully fix it.
5. It is always advised to run svn cleanup and also delete unwanted jobs from repository.
Hope it helps
Thanks
Rohan Suresh
Sabrina, Generation time taken is high for all the jobs & not specific to any particular job. Even for the jobs with minimal tComponents (~2-3) it takes longer time say around 3-5 mins. Java version - java version "1.8.0_92" Can you advice on, what could be the typical time taken for Generating & Deploying a job having minimal tComponents with no complexity/transformations. Thanks & Regards Ashok Karnam
Hi
Ashok Karnam,
The job-generation consuming time is determined by many elements, job design, data volumn, your prod environment, the component you are using and so on.
It's a little hard for us to repro your slow job-generation issue on our side.
Would you mind report a ticket for your issue on Talend Support Portal? In this way, our collegues from support team can give you a remote assistance on this performance issue.
Best regards
Sabrina
Thanks Sabrina, Job design is very simple job, populating data from source table to target table without any transformations & having bare minimal components - tMySQLInput, tMap, tMySQLOutput with a volume of 10,000 rows. Input & Output databases are within LAN. As suggested, I will create a ticket on Talend support portal for the said issue. Thanks & Regards Ashok Karnam
My experiences is, the speed of the SVN has the most influence. If you have less network latency you can increase the build speed significant. We also see under Windows it runs much slower than under Linux.
Even we face the same problem while migrating from talend 5.6.1 to 6.1.1. 6.1.1 is mush slower compared to 5.6.1 ? kindly recommend some possible solution for this problem. Thanks and regards, Amritha
