Skip to main content
Announcements
Introducing a new Enhanced File Management feature in Qlik Cloud! GET THE DETAILS!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

how to use talend open studio tHDFSList against an HDP2.6 on Centos

Hi,

i'm trying to connect to an HDFS from HDP 2.6 hosted on centos. i wanted to use on the talend open studio 6.4 the coponent tHDFSList I downloaded https://github.com/steveloughran/winutils configure the job's advanced parameter -Dhadoop.home.dir="c:/temp/" this is where i copied over the content of the 2.8.0-rc3 directory

I also tried to add -DHADOOP_OPTS=java.library.path=c:/temp/bin'

upon execution i get the WARN message :

org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using building-java classes where applicatble Exception in coponent tHDFSList_1 java.lang.IllegalArgu;entException: Can not create a Path from an empty string

Looking for a hint, on what is required to be able to create Talend jobs against CentOS based HDP 2.6? Also, i need to be able to port the jobs to a centos based Talend 6.4 running environment, so the solution needs to cope with the multiOS client plateform.

any hint/help greatly appreciated

Labels (4)
1 Reply
Jesperrekuh
Specialist
Specialist

if possible use the WebHDFS api and a tRestClient.
Less config needed.
https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html