topic Re: Talend JVM in runtime in Talend Studio

Talend JVM in runtime

Anonymous — Sat, 16 Nov 2024 09:59:31 GMT

We are using Talend Big data Enterprise edition.
In our jobs, volume of data subject to change. Sometime, we will process 1 million and other time 10 million. Job is getting failed due default JVM memory settings.
I want to change JVM memory settings in runtime, depending on the amount of data we process. Triggering the job through unix. How to handle this case while running through shell? Kindle guide me.
Thanks

Re: Talend JVM in runtime

Anonymous — Fri, 17 Mar 2017 07:43:35 GMT

Hi,
So far, you can not change JVM setting when build a job in studio.
You can e xport your jobs as Standalone Jobs(.sh or .bat) and change JVM setting from commandline.
Best regards
Sabrina

Re: Talend JVM in runtime

Anonymous — Fri, 17 Mar 2017 15:23:23 GMT

Thanks Sabrina.
Right now, our .sh has default settings as
java -Xms256M -Xmx1024M
But this need to be changed in runtime, while running the job depending on the amount of data, we are processing.
Thanks

Re: Talend JVM in runtime

Anonymous — Sat, 18 Mar 2017 15:02:36 GMT

You cannot change this setting while the job is running! It is not a lack of ability of Talend, it is simply the fact, these settings are initial settings for the JVM right at the start.
It is not a problem to allow much more memory with this parameters. The memory will only be allocated if it is needed.

Re: Talend JVM in runtime

Anonymous — Sun, 19 Mar 2017 15:42:05 GMT

Thanks jlolling.
If i run with default settings in .sh file, job fails due to out of memory issue for huge dataset.
java -Xms256M -Xmx1024M
You want us to allocate more memory(maximum anticipated depends on data in future)?. Any other way to handle this? Kindly help me.

Re: Talend JVM in runtime

Anonymous — Tue, 21 Mar 2017 07:45:07 GMT

Hi,
Have you tried to allocate more memory to your job to deal with your out of memory issue before running?
You can open the Job which you want to allocate more memory to and in the Run view, open the Advanced Settings tab and select the Use specific JVM arguments box. Then allocate more memory to the active Job by double clicking the default JVM arguments and editing them.
Best regards
Sabrina