topic java.lang.OutOfMemoryError with tDBInput in Data Quality

java.lang.OutOfMemoryError with tDBInput

Anonymous — Sat, 16 Nov 2024 05:41:36 GMT

Hello,

I have an OutOfMemoryError (Java heap space, or GC overhead limit exceeded) with tDBInput (MySQL) when I try to inner join 2 MySQL table (left w/ 10M rows and right w/ 20M rows). I have 8GB of RAM but I went to the tMap Basic settings to put my Desktop path in the "Store on disk" form. So my question is how to handle big data sets when doing "tMap" operations ?

Thank you in advance,

victorcib15.

Re: java.lang.OutOfMemoryError with tDBInput

Anonymous — Mon, 03 Jun 2019 03:41:33 GMT

Hello,

In your lookup, do you load all the columns of your table or did you select only the really required columns? Have you tried to allocate more memory to your job?

Let us know if this article helps.

https://community.talend.com/t5/Migration-Configuration-and/OutOfMemory-Exception/ta-p/21669?collapse_discussion=true&q=ava.lang.OutOfMemory&search_type=thread

Best regards

Sabrina