topic Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space in Talend Studio

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Anonymous — Mon, 14 May 2018 13:08:54 GMT

Hi All.
One Table has 4,5 million rows.
I am Exporting those rows to another table and I am getting this error message.
"Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

I read many discussions about that same problem and I applied these configurations:

+ For the cursor in the input component: 100000

+ For the advanced settings of the execution tab for the JVM parameters: -Xms1024 // -Xmx1024 // -XX:-UseGCOverheadLimit

Please find attached the design of my job.

I want to know if it's an optimization issue ( so I have to change the design of the job, ...which it's not very possible due to business model matters) or it's a classic memory issue (so I have just to increase the memory for JVM).

I'm on Talend Open Studio for Data Integration 6.5.1.

I have to wait 2 hours for each test... it's a little bit hard lol

Thanks for your time and answers!
Hedi

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

TRF — Mon, 14 May 2018 13:12:05 GMT

Did you tried to change Xms value to 2048 or 4096 or more?

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Anonymous — Mon, 14 May 2018 13:14:52 GMT

Hello TRF! Thank you for your answer and attention!

Yes I did it once... -Xms2048, but with a -Xmx245 (not sure about the value and the role of this parameter)

Hedi

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Alessandro_Lachina — Mon, 12 Apr 2021 08:33:24 GMT

Try without use cursor in input and put a minor value on destination (batch 10.000 and commit every 10.000).

I think use cursor affect memory usage, if number of rows in cursor is high could be the problem.

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

gjeremy1617088143 — Mon, 12 Apr 2021 08:48:27 GMT

Hi in JVM parameters : xms is the minimum amount of memory you want your job use at his launch, and xmx the maximum amount of memory the job will use.

one exemple : xms 1024 xmx 4096 , the job will begin with 1024 and can up to 4096 when more memory usage is needed.

Send me Love and Kudos and Greetings from France

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

kakooo16 — Mon, 12 Apr 2021 13:49:38 GMT

Salem Hedi ,

U can use store Temp Data in tmap