Hardware configuration for Talend open studio jobs

[PB1608798450]

Hello,

We are having database of 2 TB with write transaction of 100 million per day (5K tables approx). As part of migration activity we need to schedule it as job which runs 24*7. So, we need to estimate hardware requirement for same for optimal performance on both windows or linux platform. Can someone please suggest ?

Appreciate your response. Thanks in advance.

Thanks,

Palak

[Sourav_Roy]

Hi,

Kindly Ensure you have more than 8-16GB of free Memory at all times(UNIX/WIN) and the job does not go on a stale state.

Also,

Editing the memory and JVM settings will help a Lot in such a case

To gain performance at runtime and when launching Talend Studio you can edit the memory settings in the .ini file.

By default, the .ini file sets the following JVM parameters:

-vmargs

-Xms64m

-Xmx768m

-XX:MaxPermSize=512m

-Dfile.encoding=UTF-8

Note: The settings of the .ini configuration file will only impact the performance of Talend Studio, not the job execution itself.

Procedure

1. Find and open the TOS_DI-linux-gtk-x86_64.ini file.
2. Edit the memory attributes according to your system memory availability. For example:
3. -vmargs -Xms512m -Xmx1536m -XX:MaxMetaspaceSize=512m

Tip: For big projects, you may need to increase Xmx to 4096m For more details, see http://www.oracle.com/technetwork/java/hotspotfaq-138619.html.

Example

With 8 GB of memory available on 64-bit system, the optimal settings can be:

-vmargs

-Xms1024m

-Xmx4096m

-XX:MaxPermSize=512m

-Dfile.encoding=UTF-8

[PB1608798450]

thanks Sourav for your response. But will 8-16 GB of RAM will suffice for almost hundred job running simultaneously?

[Sourav_Roy]

Hi,

I personally don't think 8-16GB will suffice 100 jobs at runtime. I can suggest using atleast 32GB or more for this but there needs to be sufficiently fast Processing Power (CPU) to enable multi threading and proper utilization of RAM.

Also, usage of Disk based options where memory is involved can improve the state.

[Anonymous]

Hello,

Depending on the number of executed processes running on a module, you may need to increase the available memory

Here is online documentation about: TalendHelpCenter: Hardware requirements for talend open studio V 7.3.1

Best regards

Sabrina