Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us to spark ideas for how to put the latest capabilities into action. Register here!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

how to use talend open studio tHDFSList against an HDP2.6 on Centos

Hi,

i'm trying to connect to an HDFS from HDP 2.6 hosted on centos. i wanted to use on the talend open studio 6.4 the coponent tHDFSList I downloaded https://github.com/steveloughran/winutils configure the job's advanced parameter -Dhadoop.home.dir="c:/temp/" this is where i copied over the content of the 2.8.0-rc3 directory

I also tried to add -DHADOOP_OPTS=java.library.path=c:/temp/bin'

upon execution i get the WARN message :

org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using building-java classes where applicatble Exception in coponent tHDFSList_1 java.lang.IllegalArgu;entException: Can not create a Path from an empty string

Looking for a hint, on what is required to be able to create Talend jobs against CentOS based HDP 2.6? Also, i need to be able to port the jobs to a centos based Talend 6.4 running environment, so the solution needs to cope with the multiOS client plateform.

any hint/help greatly appreciated

Labels (4)
1 Reply
Jesperrekuh
Specialist
Specialist

if possible use the WebHDFS api and a tRestClient.
Less config needed.
https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html