Performace optimization in Talend

[Anonymous]

Hi Team,
Is there any performance optimization guide or tuning techniques for Talend. How to achieve any task to be completed in less time than required?. What is the best way for Tuning jobs to get better performance.
Any suggestions?

Regards,
Vijayvanamala

[Anonymous]

There are a lot of possibilities to optimize jobs.
Starting with database components:
Activate batch mode in output components and activate fetch size in input components.
Switch on parallization...
Mostly the performance will be influenced be a clever job design. Never try to process all available data at once, better cut is in handy chunks and process them in iterations.
Avoid huge complex SQL joins and statements, try to join data in a tMap within the job.
Normally customers think about asking a consultant for help or a good idea is to book a training by Talend!

[Anonymous]

Thanks for your Suggestions Mr.jlolling.
IS there any document designed by Talend regarding Performance Optimization?.
Because i have a job where my source is OracleDB and Source is amazon S3 and there are millions of records to be moved form source to target (something in TB's) So, to improve the performance the above steps are sufficient or anything more is required to be done?.
Regards,
Vijayvanamala

[Anonymous]

Hi,
Here is a online document about Talend Studio TalendHelpCentererformance preferences, which works on the whole studio.
For a specific job, usually the performance is most based on DB connection, job design, component setting(Check Use Batch mode option).

Best regards
Sabrina

[Anonymous]

HI Sabrina,
Thanks for sharing information about Performance tunning in Talend. Actually where I can find Component Settings(I am not using Talend Subcribed version) and how to activate batch mode in output components and also fetch size in input components.
With Warm Regards,
Viswanath.

[Anonymous]

hi all,
have a look at hint option for Oracle Output 'to improve flow data'.
regards
laurent

[Anonymous]

@vijayvanamala: To improve performance there is much more to be done as asking in a forum generic questions about "performance". You should think about a good design. After that you can implement your design in Talend and not vis versa.

[Anonymous]

@jlolling
wise advice
+1
regards
laurent

[mertez]

We have a case of a web service that for each received event triggers a lookup (tPostgresqlInput) to the database. Lookup is done with load at once option in the tMap component.
Our question is whether talend between event executions maintains its connection?
We are trying to minimize connection overhead that is done with each query. Any tips how to optimize this? Would a tConnection component make any difference?
Thank you.

[Anonymous]

Your link is not working. Suggest another link.