Increase the performance of a Talend Job

[Anonymous]

Dear all, i have a question regarding the best practices and tricks to use TOS for DI. 

Is there someone who can list all the best practices and preliminary actions to do during the creation and after it to have a perfect job?

(considering the fact that 99% i read DB2 databases)

thanks in advance for the list

Best regards

Sam Fantini

[Anonymous]

Hello,

Performance issue usually cause by the DB connection or the job design, could you upload some screenshots of your job?

Best regards

Sabrina

[Anonymous]

My question was in general, if there is a way (mix of particular components) to have the best performance in all the jobs. A sort of best practices.

[Jesperrekuh]

I totally understand your question, yes there're.

But please distinct between

A - Infrastructure / network / security (packet inspection)
B - Getting (99% are reads) db-data. Avoid conversion between types and focus on efficiency/throughput, like fields get them as number in stead of strings, so in your select be smart. Join on db-side probably indexed. As for of DB-writes... same rules apply.

C - Components/talend, if you have a lot of columns and you have to process a lot of rows, reduce overhead, get rid of columns especially strings... if you dont need them dont get them! Like tMap and many columns tend to slow down, but that's in every system.
D - Java engine and memory vs disk i/o. Multithreaded, independent jobs, parallel execution.

By (talend) design you can do whatever you want ... the deeper within your system ... the quicker ... be smart, be savvy!