topic Re: high memory usage on talend in linux in Talend Studio

high memory usage on talend in linux

indeed_1 — Fri, 15 Nov 2024 23:17:58 GMT

1-i have rhel machine with 500GB of RAM, what is the correct value of xms and other parameter?

here is the current values:

[root@machine ~]# ps -elf | grep java

0 S root 4359 61786 0 80 0 - 28203 pipe_w 21:56 pts/13 00:00:00 grep --color=auto java

0 S db 41871 55452 39 80 0 - 29738034 futex_ 16:00 pts/7 02:21:34 /bin/java -Xms512m -Xmx1536m -Dfile.encoding=UTF-8 -Dosgi.requiredJavaVersion=1.8 -XX:+UseG1GC -XX:+UseStringDeduplication -XX:MaxMetaspaceSize=512m -jar /data/db/dba/DataIntegrationPackage-CUST/talend/talendsoftware/TOS_DI-20200219_1130-V7.3.1/plugins/org.eclipse.equinox.launcher_1.5.200.v20180922-1751.jar -talendReload false -os linux -ws gtk -arch x86_64 -showsplash -launcher /data/db/dba/DataIntegrationPackage-CUST/talend/talendsoftware/TOS_DI-20200219_1130-V7.3.1/TOS_DI-linux-gtk-x86_64 -name TOS_DI-linux-gtk-x86_64 --launcher.library /data/db/dba/DataIntegrationPackage-CUST/talend/talendsoftware/TOS_DI-20200219_1130-V7.3.1/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.900.v20180922-1751/eclipse_1801.so -startup /data/db/dba/DataIntegrationPackage-CUST/talend/talendsoftware/TOS_DI-20200219_1130-V7.3.1/plugins/org.eclipse.equinox.launcher_1.5.200.v20180922-1751.jar --launcher.overrideVmargs -exitdata 6802b -vm /bin/java -vmargs -Xms512m -Xmx1536m -Dfile.encoding=UTF-8 -Dosgi.requiredJavaVersion=1.8 -XX:+UseG1GC -XX:+UseStringDeduplication -XX:MaxMetaspaceSize=512m -jar /data/db/dba/DataIntegrationPackage-CUST/talend/talendsoftware/TOS_DI-20200219_1130-V7.3.1/plugins/org.eclipse.equinox.launcher_1.5.200.v20180922-1751.jar

2-always see high memory usage on server with this command "pmap -p PID" e.g currently for talend is: total 119083212K

3-how can i change those parametere?

any idea?

thanks

Re: high memory usage on talend in linux

Anonymous — Wed, 19 Jan 2022 21:59:55 GMT

You can set the memory params for Talend Studio by following these instructions.....

https://community.talend.com/s/article/Allocating-more-memory-to-Talend-Studio-LJfwT?language=en_US

120GB memory usage is ridiculously high. I presume that you only have one instance of Talend Studio open? Can you give a bit more information as to what you are doing on Talend Studio when you see these numbers? How long has it been running? Are you running any jobs? If so, what sort of data (how much/how big in terms of size)?

This behaviour seems very odd.

Re: high memory usage on talend in linux

indeed_1 — Thu, 20 Jan 2022 10:01:38 GMT

1-yes i have one instance.

2-run a job. After jobs done i expect when i run (pmap PID of talend) it show low usage but still show 120gb used by talend. But when i run top cmmand it show java only use 0.7 of memory!

3-after jobs done it doesn’t free memory and seems talend still use too much memory continuously!

any idea?

Thanks

Re: high memory usage on talend in linux

Anonymous — Thu, 20 Jan 2022 11:27:15 GMT

I have been investigating this (I am on a Mac and don't use Linux) and have found this....

https://unix.stackexchange.com/questions/164653/actual-memory-usage-of-a-process

I don't think that 120GB is truly representative of what is actually being used by Talend here.