[resolved] java.lang.OutOfMemoryError: Java heap space talend

[_AnonymousUser]

Hi All,
I have a simple job with the following components
tMySqlInput ---->tMap---->tMySqlOutput
This is for inserting/updating the records from 1 MySql Db to another.This is working fine for a table with 125000 records .I am getting an error when tried with a table of 11587500 records.The error is:
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.util.LinkedList.listIterator(Unknown Source)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractSequentialList.iterator(Unknown Source)
at routines.system.RunStat.sendMessages(RunStat.java:261)
at routines.system.RunStat.run(RunStat.java:225)
at java.lang.Thread.run(Unknown Source)
Exception in thread "main" java.lang.Error: java.lang.OutOfMemoryError: Java heap space.
Please help.
Thanks,
scsdev

[_AnonymousUser]

Hi,
Thank you very much.It helped me.
scsdev

[Anonymous]

Hi,
Welcome, Please feel free to post your issue on talend forum.
Best regards
Sabrina

[Jackson0123]

This error java.lang.OutOfMemoryError: Java heap space indicates that NiFi is running out of memory while processing your flow. Since you are using NiFi 1.0 with around 123 processors and also seems to handle about 200 flow files of 50 MB each, the data load is quite higher than usual. Even with 4 GB of heap memory configured in bootstrap.conf, NiFi may not be able to handle this workload efficiently due to memory constraints and known inefficiencies in that version. 

Upgrading to a newer version of NiFi can be considered as the best way recommended. This is suggested because in most cases, later releases will include several memory and performance optimizations that will help you to handle such circumstances. Along with this change, consider reducing the number of concurrent tasks, processing smaller batches, setting appropriate back pressure limits, and ensuring repositories use disk storage instead of memory. Monitoring heap usage through GC logs can also help identify memory-heavy processors and prevent similar issues in the future. If you are interested in learning about the  types of OutOfMemoryError, you can check out this blog: Types of OutOfMemoryError, Causes, and Solutions.