JVM Xmx not always used during the execution

[FPellissier1625558358]

Hello,

We have a not too complex job ( mostly DB Input and Output, no look-up, only mapping, but multiples tRunJob, "use an independent process" is uncheck) which need up to 8Go to run fast (many rows to transfer). So we modify the jvm property, set in it to Xmx8G.

• When we run the build job on a computer (Java Oracle 1.8.0_202), the job run fast (< 25min) and use all the 8Go we assign to him.
• When I try the same build job on my computer (Zulu Open JDK 11.0.12, then with Java Oracle 1.8.0_202) the job is very long (I killed it after more than an hour) and doesn't use all the free memory. It stay stuck under 500Mo
• When we Publish it on our RemoteEngine (Zulu Open JDK 11.0.12), configure a run profil with Xmx8G, same issue: it takes 34 hours to run. The process detail show us the Xmx parameter is well transmitted in the run command line

I try to change the Xms and set it to Xms1G: it's take into account. The resource monitor show me that the "validation memory" goes to 1Go when I run the job. But it does not used it, it stay under 500Mo

Both local computers run Windows 10 64, on the same network or VPN (we try both).

The Talend Studio who build it was with Zulu Open JDK 11.

The Remote use RedHat.

Do someone knows what could cause such a difference?

[Anonymous]

If you are not running with the same JDK/JRE this could be causing issues. You should be running your jobs with the JDK/JRE that they are compiled with to make sure you don't get any unexpected behaviour. Not that I would expect there to be massive issues by doing this, but it is something to keep in mind.

Also network bandwidth could be an issue. Are the machines using comparable networking devices?

When you have tried with two different JVMs on the same machine, are you sure that the change of JVM actually led to that JVM being used? Were your system variables PATH, JAVA_HOME, etc configured for the change?

Was there any other activity taking place on the databases when you ran these tests?

Do the machines have equal RAM? Are other processes running which could be using the RAM?

[FPellissier1625558358]

Hello. Thanks for you answer.

We aren't in the same country for our tests. But there shouldn't have network limitation, our infrastructure team check this part.

When I tried different JVMs on my machine, I checked the PATH/JAVA_HOME and the JDK used by the process during the execution, to be sure it takes the one I want.

The machine who run fast have 20Go, mine 16Go. All other processes were killed, I follow the RAM usage and there always was room for more.

[Anonymous]

The machine that ran this job the quickest, how far away from the databases it is interacting with is it? The ones that took a long time, are they further away? Is anyone who is testing working from home? Is the fastest machine based in an office by any chance? If people are testing from home, then their bandwidth can affect the rate as well as VPN latency.

If a job is compiled, then it is dependant on the system that it runs on. If a compiled job is being passed to different machines and there is such staggeringly different performance, it is either down to the machines, the set up of the machines, the network capability of the location OR (and I forgot to mention this) the time in which the job is run. I am based in the UK and when I run a job interacting with Salesforce during my working day (up until about 3 or 4pm) it runs MUCH quicker than if I run it after that. Why? Because we have more staff in the US actively engaging with our Salesforce data (one of the reasons anyway). All of these factors need to be considered.