topic Re: Delete HDFS File Based on Creation Date in Talend Studio

Delete HDFS File Based on Creation Date

stephanie_scott — Fri, 10 Mar 2017 14:35:40 GMT

Good morning,
I'm attempting to delete files from a HDFS directory based on creation date. I'm using something similar to the solution found here:
My problem is rather than using tFileList/tFileDelete, I need to use tHDFSList/tHDFSDelete. However tFilterRow does not connect to tHDFSDelete.
I'm somewhat of a novice in Talend, teaching myself little by little. Any help or ideas on how I can change this flow to work is greatly appreciated!

Re: Delete HDFS File Based on Creation Date

Anonymous — Tue, 14 Mar 2017 03:09:14 GMT

Hi,
The tfilterrow component requires an output component. Could you please try this work flow to see if it is Ok with you?
tHDFSConnection-->tHDFSList-->tHDFSProperties-->tMap-->tFilterRow-->tfileoutputxxx-->tHDFSDelete
Best regards
Sabrina

Re: Delete HDFS File Based on Creation Date

cterenzi — Tue, 14 Mar 2017 17:23:45 GMT

If the output of tFilterRow is the list of files to delete, inserting a tFlowToIterate between it and tHDFSDelete should let you loop over the files and delete each one.