OutOfMemoryError

[Anonymous]

Hi,
I am having a job which will move data from sybase ASE to sybase IQ,
in this mapping another table is joined and a look is performed with that table, in that look up table 9 Million records,
while running this job we are facing the below mentioned issue
java.lang.OutOfMemoryError: GC overhead limit exceeded


Thanks in Advance
Siva

[Anonymous]

Hi siva,
Did you check the response in https://community.talend.com/t5/Design-and-Development/Out-of-Heap-Space/td-p/61917 to see if it is OK with you?
Best regards
Sabrina

[Anonymous]

hi all,
GC overhead limit means that your Garbage collector work "too hard & and too Often".
OPtimize your job & increase (if possible) your XmX (JVM)
Have a look at
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#par_gc.oom

regards
laurent

[Anonymous]

I have Optimized the job by storing the data in local disk and it is working fine now..

Thanks
Siva

[john_williams]

When dealing with java out of memory error, java.lang.OutOfMemoryError in a Java MIDlet, this actually gives you an alert that the application is consuming more memory than the Java Virtual Machine (JVM) can allocate. You may think that setting objects to null will regulate the process. But it's not like that always. Even though you’re setting objects to null after every iteration in your infinite loops,there is no guarantee that the Garbage Collector (GC) will immediately reclaim that memory. If you see generally in Java, whichever objects created within a loop will be collected by garbage collection action after each iteration when they lose reference or go out of scope. But still, if there are any lingering references, such as static variables, collections, or fields inside classes, in such scenarios, those objects might not be collected, which will lead to memory leaks.

Also, in any kind of resource-constrained environments like J2ME MIDlets, the garbage collector isn’t as efficient or aggressive as in modern JVMs. Consider multiple threads running infinite loops, this can quickly exhaust heap memory. Particularly when new objects are created frequently within those loops there is more chance of heap memory exhaustion. Always have an eye on any collections or static fields that are unintentionally holding onto object references.

To address this issue, you should keenly review the application’s memory management practices. You can always consider checking for unnecessary object creation inside infinite loops. And also try to reuse objects where possible. You can check for no collections or static variables to retain unused references.  Apart from this, there are a few more OutOfMemoryError scenarios that might occur. If you wish to learn about their types, causes and solutions, you can refer to this blog post.