Executing Talend Job from linux machine which can connect to HDFS

Anonymous — Sat, 16 Nov 2024 10:31:49 GMT

Hi
I am using Hadoop Open Studio for Big Data tool for developing my ETL program in Windows machine. And I set
-Dhadoop.home.dir=C:/tmp/webutils for all big data related components. After developing the job, I am placing this job (as jar file) in another Linux machine. But when executing that jar file(talend job), we are facing below issues
Exception in component tHDFSInput_1
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3106)

how can I execute the talend job from linux machine which can connect to another hadoop box and perform the job.
Thanks a lot in advance.

Prakash

Re: Executing Talend Job from linux machine which can connect to HDFS

Anonymous — Mon, 11 Jul 2016 04:39:35 GMT

Hi,
Have you tried to build jobs as a standalone job(.sh file/ .bat file) if you want to reuse the Job in Talend Studio installed on another machine?
Please have a look at document about:TalendHelpCenter:How to build Jobs.
Best regards
Sabrina

topic Executing Talend Job from linux machine which can connect to HDFS in Talend Studio

Executing Talend Job from linux machine which can connect to HDFS

Re: Executing Talend Job from linux machine which can connect to HDFS