Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello Community,
Im having serious issues with a Job.
Facts:
* Tablerows in DB: ~2 Million rows
* Select * from Table: approximately 15 seconds runtime in MSSQL Management Studio
* Job is getting 8GB of RAM
* Dfile.encoding=UTF-8
* Java Version 7, Update 79
* Talend Version: 5.2.2 - Build id: r99271-20130307-0242
* It's a job for a Dimension
* I'm having some Jobs, which are running with 60k+ rows/second
Job is starting at 1500 Rows / Second and is immediately losing 50% and then steadily dropping, till it hits ~ 100 rows / second.
I've tried so far:
* setting an offset with a limitation of 10k / 100k rows
* removing nearly all the selected fields in the statement
* removing nearly all the fields in the tMap component
* setting new indexes for my DB-Table
* Splitting the Job into small pieces (200k rows each job)
Any idea how I can "debug" this problem - to find and resolve the bottleneck?
I'd appreciate any help - my Boss is getting really troublesome
D_ARTIKEL_… .zip