General: Troubleshooting Jobs

Anonymous — Thu, 22 Sep 2016 07:54:26 GMT

Hello Community,

I'm also new to Talend. I have a question for Troubleshooting Jobs.
Some Jobs run at 30k rows/sec.
My new Job (2 million rows, over 100 Type 1 SCD Fields, several Type 2 SCD Fields) starts really good with 2k rows/sec - but it's rate is dropping pretty hard after a while. It goes down to like 150 rows/sec.

* The Job is using actively 6-7 GB of RAM
* Parameters Dfile.encoding=UTF-8 / Xms500M / Xmx8192M are set
* Before weeks, the Job did run pretty smooth! (New change was adding more fields) - But after deleting them again, no change.
* dom4j parser
* Java 1.6
* SQL-Server where the Data is coming from: He's bored!
* SQL-Server where the Data is going in: He's bored!
* Talend version: 5.2.2 r99271-20130307-0242
* I think I have to use exactly this version, due the ERP-Systems Company approval
* I was also trying to run the Job with the .jar - no difference, slow..

So - How do you guys start Troubleshooting Jobs to increase performance?

Thank you for any ideas/informations and best regards
dit0x

Re: General: Troubleshooting Jobs

Anonymous — Thu, 22 Sep 2016 08:41:51 GMT

Update:
The Job runs now with 6k rows/sec - Why? I dont know! NOTHING changed.
Next time this happens, I'm going to check the network rates for reads and writes and give you an update.

topic Re: General: Troubleshooting Jobs in Talend Studio

General: Troubleshooting Jobs

Re: General: Troubleshooting Jobs