topic Re: TIS enterprise edition - load balancing in Talend Studio

TIS enterprise edition - load balancing

Anonymous — Sat, 16 Nov 2024 12:56:28 GMT

Hi,
I hope this is the right forum to ask this question.
We currently have a job that we parallelize to utilize the 8 cpus on one machine (iterate with parallel excution and tparallelize).
If you setup a virtual server that contains two machine each with 8 CPU's - If you assign a job to run on the virtual server will talend utilize all 16 cpu's if needed?
i.e can we do iterate with 16 parallel executions on the virtual server.
I am just trying to understand how the load balancing works.
Thanks,

Re: TIS enterprise edition - load balancing

Anonymous — Thu, 05 May 2011 16:39:43 GMT

Hi
As I known, it always select the optimum efficiency CPU from 16 CPU to execute the job and regardless of how the composition of job, means that there only one CUP be used to execute the job.
Best regards
Shong

Re: TIS enterprise edition - load balancing

Anonymous — Thu, 05 May 2011 17:30:22 GMT

Hi Shong,
I think I understand what you are saying. It sounds like the load balancing will pick the most optimal server to run on based on the virtual pool definition rather than see the combined servers as a resource.
So if we have a job with an iteration (x8 parallel) and the virtual pool had one 8cpu machine and one 4 cpu machine - the load balancing will choose to run the job on the 8 cpu server?
Is that correct?

Re: TIS enterprise edition - load balancing

Anonymous — Mon, 09 May 2011 17:14:56 GMT

Any clarifications on my question??
Basically it seems you cannot split a single process across mutliple servers - correct me if I am wrong.
Thanks,

Re: TIS enterprise edition - load balancing

Anonymous — Mon, 09 May 2011 23:00:42 GMT

Hi,
Your assumptions are correct.
You can also take advantage of you 8+8 cpu with the MPx edition that will take advantage of a hadoop framework.
This time, it will split the data into a distributed file system and ask each node to process part of the data
Benjamin

Re: TIS enterprise edition - load balancing

Anonymous — Tue, 10 May 2011 19:10:08 GMT

Thanks Benjamin.