topic Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space in Talend Studio

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

amtkmr1990 — Thu, 10 Sep 2020 09:16:09 GMT

Hi Team,

I getting the following error for a job. Attached is the job

tMSSQLINPUT -> tmap -> tlogrow

Table rows - 28595

Table columns - 246

Could you please suggest solution. Tried lot of google and talend community but didn't work.

Error : Message

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Unknown Source)

at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)

at java.lang.AbstractStringBuilder.append(Unknown Source)

at java.lang.StringBuilder.append(Unknown Source)

at customer_hub_integration.test_account_0_1.test_account$1Util_tLogRow_1.format(test_account.java:8062)

at customer_hub_integration.test_account_0_1.test_account.tDBInput_1Process(test_account.java:16018)

at customer_hub_integration.test_account_0_1.test_account.runJobInTOS(test_account.java:16307)

at customer_hub_integration.test_account_0_1.test_account.main(test_account.java:16138)

[statistics] disconnected

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

JohnRMK — Thu, 10 Sep 2020 10:26:46 GMT

Hello,

When you read the data from the database, it is sent to ram memory. So if you don't have enough capacity on your pc, it's normal to have this error.

However here is a small solution: In Advanced Setting of the Run view, you increase the memory allocated to the job (Xmx). by default it's 500 MB so, it's up to you to see what interests you.

Also, if you do any transformation in the tMap, you can store the data in the hard disk instead of the ram, in the adanced setting of the tMap, you have an option to choose a temporary disk. if you don't do anything with the tMap, you can delete it.

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

amtkmr1990 — Thu, 10 Sep 2020 11:05:48 GMT

1. Ok Let me give try to tmap temp disk write.

2. i have 16 GB RAM

3. Talend is showing to occupy 2.5 GB in task manger

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

amtkmr1990 — Thu, 10 Sep 2020 11:44:53 GMT

I am getting the same error

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

amtkmr1990 — Thu, 10 Sep 2020 11:46:11 GMT

if i read 1000 rows its working

How can read 28595 rows in multiples of 1000 rows.

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Anonymous — Fri, 11 Sep 2020 08:33:26 GMT

Hello,

The root issue in this error is that not enough heap is being given. You need to adjust the heap size for the job to a larger size.

Here is a community knowledge: https://community.talend.com/s/article/OutOfMemory-Exception-WmtmQ

Let us know if it helps.

Best regards

Sabrina

Re: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

harvyjackk — Fri, 11 Feb 2022 07:28:04 GMT

Usually, this error is thrown when the Java Virtual Machine cannot allocate an object because it is out of memory, and no more memory could be made available by the garbage collector.

Therefore you pretty much have two options:

Increase the default memory your program is allowed to use using the -Xmx option (for instance for 1024 MB: -Xmx1024m)

Modify your program so that it needs less memory, using less big data structures and getting rid of objects that are not any more used at some point in your program

Increasing the heap sizeis a bad solution, 100% temporary. It will crash again in somewhere else. To avoid these issues, write high performance code:

Use local variables wherever possible.
Make sure you select the correct object (EX: Selection between String, StringBuffer and StringBuilder)
Use a good code system for your program(EX: Using static variables VS non static variables)
Other stuff which could work on your code.
Try to move with Multy Threading