topic Re: java.lang.OutOfMemoryError: Java heap space in Talend Studio

java.lang.OutOfMemoryError: Java heap space

hgromek — Fri, 26 Jun 2020 16:55:20 GMT

I have been getting this error often with the past few jobs I have created. I am attempting to simply use a database input to an excel file. It seems like every time I use an advanced Excel output (updating file and not creating new file).

Here's my Job (my other ones are very similar), this one only brings in 20K records.

I have increased the memory for this job and enabled stream but did not fix the issue.

I have attached an image of my Job -

Here's the Java Error

Starting job SpaceHeap at 11:46 26/06/2020.

[statistics] connecting to socket on port 3824
[statistics] connected
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.getNodeObject(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.DeferredElementNSImpl.synchronizeData(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.ElementNSImpl.getNamespaceURI(Unknown Source)
at org.apache.xmlbeans.impl.store.Locale.loadNode(Locale.java:1420)
at org.apache.xmlbeans.impl.store.Locale.loadNodeChildren(Locale.java:1403)
at org.apache.xmlbeans.impl.store.Locale.loadNode(Locale.java:1445)
at org.apache.xmlbeans.impl.store.Locale.loadNodeChildren(Locale.java:1403)
at org.apache.xmlbeans.impl.store.Locale.loadNode(Locale.java:1445)
at org.apache.xmlbeans.impl.store.Locale.loadNodeChildren(Locale.java:1403)
at org.apache.xmlbeans.impl.store.Locale.loadNode(Locale.java:1445)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1385)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1370)
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:370)
[statistics] disconnected
at org.apache.poi.POIXMLTypeLoader.parse(POIXMLTypeLoader.java:164)
at org.openxmlformats.schemas.spreadsheetml.x2006.main.WorksheetDocument$Factory.parse(Unknown Source)
at org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:226)
at org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:218)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.parseSheet(XSSFWorkbook.java:443)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:408)
at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:169)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:300)
at de.jlo.talendcomp.excel.SpreadsheetFile.initializeWorkbook(SpreadsheetFile.java:447)
at demo.spaceheap_0_1.SpaceHeap.tFileExcelWorkbookOpen_1Process(SpaceHeap.java:1141)
at demo.spaceheap_0_1.SpaceHeap.runJobInTOS(SpaceHeap.java:2162)
at demo.spaceheap_0_1.SpaceHeap.main(SpaceHeap.java:2019)
Job SpaceHeap ended at 11:46 26/06/2020. [exit code=1]

Re: java.lang.OutOfMemoryError: Java heap space

Anonymous — Sun, 28 Jun 2020 06:56:22 GMT

Hello,

As you are using a custom component tFileExcelWorkbookOpen written by talend community user, could you please let us know if this related topic helps?

https://community.talend.com/t5/Design-and-Development/resolved-tFileExcelWorkbookOpen-Error-while-in-memory-saving/m-p/70447

We will appreciate it a lot if you could post your setting of JVM parameters screeshot here.

Best regards

Sabrina

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:03:22 GMT

Hi - It will happen if I use the standard Excel Output as well. here is a screenshot of my JVM Settings.

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:03:22 GMT

Hi - It will happen if I use the standard Excel Output as well. here is a screenshot of my JVM Settings.

Re: java.lang.OutOfMemoryError: Java heap space

manodwhb — Mon, 29 Jun 2020 13:12:04 GMT

@hgromek ,have you verified the below link? if not please and modify the parameters and see.

https://community.talend.com/t5/Migration-Configuration-and/Out-of-Memory-error-when-reading-large-Excel-files-in-Talend/ta-p/203437

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:18:40 GMT

it is not an Excel Input, the input is coming from a MySQL database, the Excel is used as an output.

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:18:40 GMT

it is not an Excel Input, the input is coming from a MySQL database, the Excel is used as an output.

Re: java.lang.OutOfMemoryError: Java heap space

manodwhb — Mon, 29 Jun 2020 13:34:56 GMT

@hgromek , can you try to increase JVM and check how is it working?

Re: java.lang.OutOfMemoryError: Java heap space

manodwhb — Mon, 29 Jun 2020 13:34:56 GMT

@hgromek , can you try to increase JVM and check how is it working?

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:40:56 GMT

If I increase the jvm size to this the below - I get this error. 4096M can't be handled by my machine.

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 29 Jun 2020 13:40:56 GMT

If I increase the jvm size to this the below - I get this error. 4096M can't be handled by my machine.

Re: java.lang.OutOfMemoryError: Java heap space

manodwhb — Tue, 30 Jun 2020 17:43:06 GMT

@hgromek , you need to increase to more. but you need to have hardware too.

Re: java.lang.OutOfMemoryError: Java heap space

manodwhb — Tue, 30 Jun 2020 17:43:06 GMT

@hgromek , you need to increase to more. but you need to have hardware too.

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Wed, 01 Jul 2020 14:26:25 GMT

I am working with a 8 GB, 64-bit machine. I thought I would be able to use the below, but I always get errors?

-vmargs
-Xms1024m
-Xmx4096m
-XX:MaxPermSize=512m
-Dfile.encoding=UTF-8

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Wed, 01 Jul 2020 14:26:25 GMT

I am working with a 8 GB, 64-bit machine. I thought I would be able to use the below, but I always get errors?

-vmargs
-Xms1024m
-Xmx4096m
-XX:MaxPermSize=512m
-Dfile.encoding=UTF-8

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Mon, 03 Aug 2020 14:52:53 GMT

Any other suggestions? I get the error a lot more often now, on datasets only 16K rows?

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Tue, 11 Aug 2020 16:14:42 GMT

Hey - Following up.

My Machine has the capabilities to run -Xmx4096m but Talend errors out. Also, memory error comes when I use FileOutputExcel to append data to existing file. Running it while just creating new file works fine.

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Thu, 03 Dec 2020 13:56:16 GMT

The link has not worked. If I added more RAM to my machine, would that increase the memory power?

Re: java.lang.OutOfMemoryError: Java heap space

hgromek — Thu, 03 Dec 2020 16:07:04 GMT

If I added more RAM to this machine from 8- 12GB. Would that increase memory available?

Re: java.lang.OutOfMemoryError: Java heap space

Anonymous — Fri, 04 Dec 2020 07:37:28 GMT

@Hallie Gromek, make sure you have installed the right java for your machine, if it is a 64-bit machine, you should install 64-bit Java.

Refer to this KB article to learn more about this error, all open applications will access and use RAM, go to check how much memory available before you configure the JVM parameters.

Regards

Shong