OutOfMemoryError : Java heap space / GC overhead limit exceeded

Anonymous — Sat, 16 Nov 2024 10:35:59 GMT

Hi !

I am writing a job which reads data from Excel file, inserts it in DB (temp table) and then extracts data from the DB and writes it into another Excel files.
For example, the input excel file contains employee names, and the output file is filled with many infos for each employee.
I did that process 3 times, for 3 tabs of the input file (3 different types of data).

You can see on the screenshot the first part of the process. (Of course, I have a prejob and postjob to open/close the connection, create temp tables, and stuff like that.)

The job works for small input files. But I need to process big ones... and I'm facing 'Out Of Memory' errors.
The first one was Java heap space.
Then, I configured my components to store temp files, use less memory... (For example, my tUniqRow components)
But now I have a GC overhead limit exceeded.

Note : I changed the -Xmx parameter to -Xmx4096M.

It seems that when I try to avoid Java heap space, I get the GC overhead error, and when I try to reduce the GC operations, I get the Java heap space back...

Any idea to help me please ?

Thank you !

Anth224

Re: OutOfMemoryError : Java heap space / GC overhead limit exceeded

Anonymous — Tue, 14 Jun 2016 08:59:50 GMT

Hi,
On which build version you got this issue? Have you tried to allocate more memory for your current job?
Here is a document about: TalendHelpCenter:Exception outOfMemory
Best regards
Sabrina

topic OutOfMemoryError : Java heap space / GC overhead limit exceeded in Talend Studio

OutOfMemoryError : Java heap space / GC overhead limit exceeded

Re: OutOfMemoryError : Java heap space / GC overhead limit exceeded