I'm also new to Talend. I have a question for Troubleshooting Jobs. Some Jobs run at 30k rows/sec. My new Job (2 million rows, over 100 Type 1 SCD Fields, several Type 2 SCD Fields) starts really good with 2k rows/sec - but it's rate is dropping pretty hard after a while. It goes down to like 150 rows/sec.
* The Job is using actively 6-7 GB of RAM * Parameters Dfile.encoding=UTF-8 / Xms500M / Xmx8192M are set * Before weeks, the Job did run pretty smooth! (New change was adding more fields) - But after deleting them again, no change. * dom4j parser * Java 1.6 * SQL-Server where the Data is coming from: He's bored! * SQL-Server where the Data is going in: He's bored! * Talend version: 5.2.2 r99271-20130307-0242 * I think I have to use exactly this version, due the ERP-Systems Company approval * I was also trying to run the Job with the .jar - no difference, slow..
So - How do you guys start Troubleshooting Jobs to increase performance?
Thank you for any ideas/informations and best regards dit0x
Update:
The Job runs now with 6k rows/sec - Why? I dont know! NOTHING changed.
Next time this happens, I'm going to check the network rates for reads and writes and give you an update.
