topic Re: Creating Talend Cluster Environment in Talend Studio

Creating Talend Cluster Environment

Anonymous — Sat, 16 Nov 2024 13:04:24 GMT

Hi,
I'm currently working to create a highly available cluster for our Talend environment. I have some ideas how this could work, but need some advice.
Currently, here is our environment:
Location A:
admin server
jobserver 1
jobserver 2
repository server (mysql and svn)

Location B:
jobserver 1
jobserver 2
I'd like to replicate the admin and repository server in location B. That way, if location A is completely down, all jobs would still process. Has anyone done this? If so, what did you do?
My current plan involves creating an admin and repository server in location B. The repository server would use mysql master-master replication and svn sync (location A master to B slave only). The admin server in location B would connect to the repository server in location B. All clients (TIS) would connect to location A. In the event of an outage, no new development would occur (that's OK), but at least the job scheduler would run.
So basically my environment would look like this:
Location A:
admin server
jobserver 1
jobserver 2
repository server (mysql master-master replication and svn)

Location B:
admin server (pointing to repository in location B)
jobserver 1
jobserver 2
repository server (mysql master-master replication and svn sync from location A)
Anyways, does that plan seem like it would work? Has anyone else successfully deployed a Talend cluster?

Re: Creating Talend Cluster Environment

Anonymous — Wed, 02 Mar 2011 01:46:12 GMT

Hi
About the tomcat(admin server) cluster, it is documented in TalendIntegrationSuite_IG_xxx_EN.pdf 3.3 chapter. Job Server : Adressed by Virtual Servers. (see 3.2 chapter in TAC_xxx.pdf)
Best regards
Shong

Re: Creating Talend Cluster Environment

Anonymous — Wed, 02 Mar 2011 22:02:44 GMT

Hi
About the tomcat(admin server) cluster, it is documented in TalendIntegrationSuite_IG_xxx_EN.pdf 3.3 chapter. Job Server : Adressed by Virtual Servers. (see 3.2 chapter in TAC_xxx.pdf)
Best regards
Shong

Thanks Shong. What about the MySQL server and the SVN server?

Re: Creating Talend Cluster Environment

Anonymous — Thu, 03 Mar 2011 00:29:03 GMT

Hi
Sorry, I don't have experience on Mysql server and SVN server cluster. ;( you can post your question on their website.
Best regards
Shong

Re: Creating Talend Cluster Environment

Anonymous — Thu, 03 Mar 2011 16:05:35 GMT

Well, I've managed to setup replication the SVN part using svnsync (Master-Slave only). And I can setup MySQL master-master replication pretty easily.
My main questions are around how Talend would interact with the environment. I was hoping to point the Admin server in location A, to the MySQL and SVN server in location A, and the Admin server in location B to the MySQL and SVN in location B. This way a failure of either location would not prevent jobs from being ran

Re: Creating Talend Cluster Environment

Anonymous — Thu, 12 May 2011 18:40:56 GMT

My preference has been to use Heartbeat,Corosync,and pacemaker which are the Linux HA packages.