Wait for File on a hdfs directory

Teguard_Dev · ‎2019-09-11

Hello,

I'm using Talend Studio Big Data in the licensed version and am currently trying to switch over from a filesystem based ETL Process to HDFS based.

As of now, I'm using Wait for File to watch a File Directory for new Files to trigger my ETL Process / trigger a Input Row, as seen below.

Is there a way to replicate this using hdfs?

I'd be grateful for any Insights and help.

Cheers

manodwhb · ‎2019-09-12

@KS_UKHD , you can use tHDFSExists,check the below link to know more about this component.

Teguard_Dev · ‎2019-09-12

Thanks for the response,

but isn't that Component only for checking a single File?

Could you give an example how to use this to overwatch a directory and react to Files being created?

Cheers