topic [resolved] List all the directory for a given HDFS path in Talend Studio

[resolved] List all the directory for a given HDFS path

Anonymous — Sat, 16 Nov 2024 09:56:39 GMT

Hi,
I need your help!!!.
I want to list all the directory/folder for the given HDFS path. So I used the following job

Afte I run the job I got only below log.
connecting to socket on port 4067
connected
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
disconnected
Job Read_HDFS_file ended at 20:25 02/04/2017.

I expected the list of folder in the tlogrow output, but got nothing. please see the above log. Please let me know If I miss something here..

Re: [resolved] List all the directory for a given HDFS path

Anonymous — Mon, 03 Apr 2017 04:04:17 GMT

Can you post tHDFSList properties ?

Re: [resolved] List all the directory for a given HDFS path

Anonymous — Mon, 03 Apr 2017 04:38:17 GMT

Can you post tHDFSList properties ?

Here is the tHDFSList screen shot..
Under "data/stage/retail" path, I have folders (example datafolder1, datafolder2..etc). I want to print these folder names only in tlogrow. To understand it better, I have simplified the requirement.

FYI, I tried to list the names from unix box using similar approach and it worked well.

Re: [resolved] List all the directory for a given HDFS path

Anonymous — Mon, 03 Apr 2017 22:23:21 GMT

Hi,
I resolved it myself. I removed "*.*" from filemask and then it worked well.