topic Re: HDFS Path Does not exist in Talend Studio

HDFS Path Does not exist

Anonymous — Sat, 16 Nov 2024 06:33:20 GMT

Hi Team,

While designing talend job i am facing path does not exist error (ex : /data/tbl_daily does not exist).

Here actually i am reading a text file with tHDFSInput , there i have multiple HDFS Directory paths.

all the paths exist really except one path. By using tHDFSList component i am reading the files with file mask and copying to another HDFS Path. But when i reach one path i am facing path does not exist error(which really not exist)

Job Flow :

tHDFSInput ---> tHDFSList --> tHDFSCopy

Please help me how to skip that path which is not exist really.

Thanks in advance.

Re: HDFS Path Does not exist

Anonymous — Wed, 20 Feb 2019 07:34:13 GMT

Hi,

Did you print the output of tHDFSList and see whether the values are correct? It seems you are not getting values for all the paths.

Also I did not get the reason why you are reading file from tHDFSInput and then do the tHDFSCopy? Are you storing all the paths in one of the HDFS files and then pass this value to tHDFSList and tHDFSCopy? A screenshot will be helpful to understand your use case better.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: HDFS Path Does not exist

Anonymous — Wed, 20 Feb 2019 07:49:32 GMT

Yes Nikhil, you are correct. I am storing all the paths in a file and reading those paths and copying. in the paths file table name folder is a context variable. So Some times that table name may be not be available in some paths. So, that is the reason i want to skip that path where table name directory is not available. Hope my reply was clear. Let me know for more details

Re: HDFS Path Does not exist

Anonymous — Wed, 20 Feb 2019 08:07:17 GMT

Hi Nikhil.

Here i have attached the Job screenshot. Please check once.

Talend_Job.jpg

Re: HDFS Path Does not exist

Anonymous — Wed, 20 Feb 2019 08:26:29 GMT

Hi,

If you want to check if a specific file or directory is present in HDFS path, could you please try tHDFSEXist Component?

It will resolve the issue in your case and based on the result from this component, you can configure downstream systems 🙂

Please refer the details below for details regarding this component.

https://help.talend.com/reader/mjoDghHoMPI0yuyZ83a13Q/OQ23JC_n57m6_4wookXFjA

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂