topic Heap memory increases continuously in Talend Studio

Heap memory increases continuously

Anonymous — Sat, 16 Nov 2024 11:19:03 GMT

Hi all,
I am facing a strange memory behaviour.
I have a TOS running on an Unix server :

Redhat 6.3
Memory 8 gb
Memroy parameter in karaf-wrapper.conf
# Initial Java Heap Size (in MB)
#wrapper.java.initmemory=3
# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=4096
My TOS is configured with 1 service (2 operation). The SOAP xml message is very light (435 octet). The memory is increasing continusly; The garbage collector seems not working.
Is it a normal behaviour ? Is there a parameter allow a better memory management?
Attached the heap memory from jconsole.exe
Thank you for your help

Re: Heap memory increases continuously

Anonymous — Thu, 19 Feb 2015 13:48:23 GMT

not sure why this is happing but you can do one test by which we can say this happing because of Talend job.

export job.
use .sh file for run.
and check how much memory it has been consumed by job.

Re: Heap memory increases continuously

Anonymous — Thu, 19 Feb 2015 14:12:40 GMT

Hi umeshrakhe,

How can I run the job with .sh ?
Do you want to know how many memories are used by the job (without any solication) ? As this job is a service, the memory used is depending on the numbers of calls received. Even if the job is not taking so much memory, the actual utilisation is maybe normal.
Thank you

Re: Heap memory increases continuously

Anonymous — Thu, 19 Feb 2015 17:12:42 GMT

I am in case of memory leak. This problem was not get in a pre production server.
JVM version :
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

Re: Heap memory increases continuously

Anonymous — Thu, 19 Feb 2015 17:47:38 GMT

You have to distinct between the memory the studio takes and the memory the job takes.
If you run a job from the studio it will run in its own JVM and you should be able to see this process in the process list.
The studio takes more and more memory until the maximum (configured in the ini file) is reached and works with this memory.
You job should not do the same! Normally Talend jobs are fairly small and do not need such amount of memory.

Re: Heap memory increases continuously

Anonymous — Thu, 19 Feb 2015 19:01:44 GMT

Hi Jlolling,
Thank you for your answer.
Indeed, the Talend studio use its own jvm.
In my case, I have done the test in pre production server with SOAPUI calls.
Now, when I have basculed in production environment (same configuration) and received real SOAP requests, the memory behavior becomes strange.
For more precision, I have identified the memory which is causing the issue : it's the PS OLD GEN in Heap memory.
Do you think that the GC will be perform when the PS OLD GEN reach the max memory ? Or I have to do it ? What is the normal running ?

Re: Heap memory increases continuously

Anonymous — Sat, 21 Feb 2015 08:58:27 GMT

Hi all,
I give you a feedback on my experience.
The HEAP OLD gen memory has grown to his max at 2.8 Go. After that, she has been liberated automatically.
Now, this memory is working better : it oscillate between 50 and 70 mb with regular releases.
I don't understand the memory behaviour changement.

Re: Heap memory increases continuously

Anonymous — Sat, 21 Feb 2015 10:18:23 GMT

This is very typical and nearly every systems with a garbage collection mechanism works this way. I bet you jobs works also with less memory (I would say the half is enough).

Re: Heap memory increases continuously

pgruenbeck — Wed, 24 Oct 2018 23:58:02 GMT

We found setting incremental garbage collection helps performance.

JAVA_OPTIONS = -Xmx4096m -Xms1024m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=80