Automatic Namenode failover - When using tHDFS connection

[Anonymous]

Friends,

I am new to Talend, so might be a basic question. I am using Talend 7.0.1

When there is a failover of the NameNode server, I am trying to understand the possibilities on how to make the Talend job still work (with out any changes). To make this possible, How do we define the HDFS connection? Should we do any Parameters setting? What configuration to be taken care?

Thanks for taking time, to help me understand the solution

Regards

Giri 

[Anonymous]

Hi Giri
If you have a connectivity issue to HDFS, you can't do any operations related to HDFS. Do you want the job continue to perform other business processing even though there is a connectivity failover on tHDFSConnection?

Regards
Shong

[Anonymous]

Hi Shong.

Thanks for your reply.

It is not a connectivity issue. tHDFS is configured with primary/active namenode. When there is a swtich (Cloudera) on Namenode server passive/secondary becoming active node. We have to change the configuration of the Talend job to point to the new name node.

I am trying to understand, if there is a way where we can build the Talend job so that, we dont have to change the configuration every time there is a change in the Namenode.

Regards

Giri

[Anonymous]

yes, you should configure the parameter with context variables, so that you can load the value of variables from file/DB in the beginning of job, or specify the value of variable before you execute the job script out of studio, don't need to open studio and modify the job.

[Anonymous]

We are already performing this. Every time there is a name node change, we have to modify this param entries. This option is happening frequently, so it is not wise to change the file, where ever there is a switch. I am trying to understand if there is any other option, where we dont have to change anything in the talend job/param/config file.