topic Re: tMemoryMonitor in Talend Studio

tMemoryMonitor

Anonymous — Sat, 16 Nov 2024 11:38:36 GMT

I've added this component to the Talend Exchange which I hope is a useful utility for monitoring memory of a running Job and, hopefully, helping with good memory management - avoiding Out of Memory Exceptions and also not wasting it.
All feedback welcome.
http://www.talendforge.org/exchange/?eid=1202&product=tos&action=view&nav=1,1,1

Re: tMemoryMonitor

Anonymous — Wed, 21 May 2014 08:09:40 GMT

Hi tal00000
It is a great component that helps us to monitor the memory usage of a running job. It will be better if the component can output the memory information to a log file.
Shong

Re: tMemoryMonitor

Anonymous — Thu, 22 May 2014 12:59:21 GMT

Hi tal00000
It is a great component that helps us to monitor the memory usage of a running job. It will be better if the component can output the memory information to a log file.
Shong

Thanks for the feedback.
That shouldn't be too difficult...
I'm also amending to show peak usage; although this can be already derived from the history.
Of course, it can only show the peak that is recorded at the time of sampling; but still useful.

Re: tMemoryMonitor

Anonymous — Fri, 13 Jun 2014 23:38:03 GMT

Hi, the problem with memory leaks is the error is mostly not related to the component causing the problem.
I use a standalone damon which connects via JMX to the jobs and gets the memory state without the need of add something to the job.

Re: tMemoryMonitor

_AnonymousUser — Wed, 09 Sep 2015 11:32:06 GMT

This is a really useful component. Excuse my ignorance but what is the output telling me?
tMemoryMonitor_1: using 2347MB/3217MB (max=3641MB/64%) (processors=2)
<Name of the component>: <current memory use>/<?> (<max memory use allowed>) (<number of processors>)
The value after the "/" what is that?

Re: tMemoryMonitor

Anonymous — Thu, 10 Sep 2015 05:46:49 GMT

Hi clarkec21
Read the description of this component on Talend Exchange:

Monitor the memory usage of your running Job, with tMemoryMonitor
Information on the used, total and maximum (see -Xmx) memory of the Java runtime, are written to standard output or, if a warning threshold is reached, to standard error.
Options include ‘Enabled’, ‘Interval’ (milliseconds) and ‘Warning Threshold Percent %’.
Used in conjunction with your own diagnostic messages, this component will help you to identify where memory is being used within your Job an help you to avoid Out of Memory (OOM) Exceptions.
As well as ensuring that your Job has enough memory, you’ll also want to make sure that you do not waste system memory. The Talend Job defaults are -Xms256MB and -Xmx1024MB. As well as some of your Jobs requiring more that 1024MB, some will require less that 256MB. This utility will allow you to observe the peak memory usage and to tune accordingly.
For more information on Java Runtime memory reporting, read http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html.
Used memory is totalMemory - freeMemory. Percentages are based on used memory vs. maxMemory.

In your case, 2347MB is used memory, and 3217MB is total memory.
Best regards
Shong

Re: tMemoryMonitor

ananyasingh — Tue, 15 Dec 2015 08:55:27 GMT

Hi tal00000,
It's undoubtedly a very useful component to monitor the memory usage of a job at run time. But I need to know one thing that how one can fetch the information of this component in a file, it would be great if you can help me out.
Thanks again!
Best Regards
Ananya

Re: tMemoryMonitor

Sanjay5 — Wed, 05 Apr 2017 18:56:33 GMT

Can some please explain me on what basis tMemoryMonitor calculates memory. In below example we see used, total and maximum memory has some value, so i wanted to understand on what basis these values are calculated.
tMemoryMonitor_1: using 2347MB/3217MB (max=3641MB/64%) (processors=2)