Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi !
I am writing a job which reads data from Excel file, inserts it in DB (temp table) and then extracts data from the DB and writes it into another Excel files.
For example, the input excel file contains employee names, and the output file is filled with many infos for each employee.
I did that process 3 times, for 3 tabs of the input file (3 different types of data).
You can see on the screenshot the first part of the process. (Of course, I have a prejob and postjob to open/close the connection, create temp tables, and stuff like that.)
The job works for small input files. But I need to process big ones... and I'm facing 'Out Of Memory' errors.
The first one was Java heap space.
Then, I configured my components to store temp files, use less memory... (For example, my tUniqRow components)
But now I have a GC overhead limit exceeded.
Note : I changed the -Xmx parameter to -Xmx4096M.
It seems that when I try to avoid Java heap space, I get the GC overhead error, and when I try to reduce the GC operations, I get the Java heap space back...
Any idea to help me please ?
Thank you !
Anth224
I am writing a job which reads data from Excel file, inserts it in DB (temp table) and then extracts data from the DB and writes it into another Excel files.
For example, the input excel file contains employee names, and the output file is filled with many infos for each employee.
I did that process 3 times, for 3 tabs of the input file (3 different types of data).
You can see on the screenshot the first part of the process. (Of course, I have a prejob and postjob to open/close the connection, create temp tables, and stuff like that.)
The job works for small input files. But I need to process big ones... and I'm facing 'Out Of Memory' errors.
The first one was Java heap space.
Then, I configured my components to store temp files, use less memory... (For example, my tUniqRow components)
But now I have a GC overhead limit exceeded.
Note : I changed the -Xmx parameter to -Xmx4096M.
It seems that when I try to avoid Java heap space, I get the GC overhead error, and when I try to reduce the GC operations, I get the Java heap space back...
Any idea to help me please ?
Thank you !
Anth224