topic Re: tWaitForFile for HDFS ? in Talend Studio

tWaitForFile for HDFS ?

rohitpatil1993 — Sat, 16 Nov 2024 01:08:46 GMT

Hello People

My requirement is to wait for a file in HDFS and move to next step of flow only when the file appears/exists in directory.

If the file is not yet present when the flow run, It should loop again until the file does not appear. One the file appear, then the flow moves ahead.

Will tWaitForFile work for this scenario ?

Is there any other approach to achieve this functionality ?

Thanks in advance.

Re: tWaitForFile for HDFS ?

Anonymous — Mon, 16 Nov 2020 02:23:32 GMT

tWaitForFile is used monitor the local file system. You can try tLoop (while type)+tHDFSFileExist, define a context variable (boolean type) and set its default value as true, once the file exists, set the context variable as false and exit the loop, use tSleep to pause for a few seconds between each cycle.

Regards

Shong

Re: tWaitForFile for HDFS ?

rohitpatil1993 — Fri, 18 Jun 2021 09:28:06 GMT

Worked as expected.

Here is the solution for those who might need it.

Re: tWaitForFile for HDFS ?

Anonymous — Fri, 18 Jun 2021 09:31:03 GMT

Great! Although your feedback is late. 😉