How to process files only for the past hour?

[Anonymous]

I have continuous sensor data coming in every 5 mins in form of files. I want to pick files only for the past hour and do the required processing. for e.g: talend job runs at 12:01pm , it should pick all the files from 11:00 am to 12:00 pm only.
Can anyone please suggest the approach I should take to make this happen within talend.

[Anonymous]

Hi,

Could you please elaborate your case with an example with input and expected output values?

Best regards

Sabrina

[vapukov]

if file name include date time pattern, just use

tFileList -> iterate

in tFileList, use pattern like:

"commonpart-*"+TalendDate.formatDate("yyyyMMddHH",TalendDate.addDate(TalendDate.getCurrentDate(),-1,"HH"))+"*.csv"

if file names not include time pattern - need work with tSystem and bash, look there

https://stackoverflow.com/questions/26968159/list-of-files-modified-1-hour-before

than parse result

[Anonymous]

I had initiated a post on https://stackoverflow.com/ .

https://stackoverflow.com/questions/45550213/how-to-process-files-only-for-the-past-hour-using-talen...

I have reached to a point but facing issues(in comments)

[vapukov]

no comments in thread (as I can see - You are move it to chat)

but in any case - suggested solution, no cover previous Hour,

If You check 1 Hr from current moment -  it could have or duplicates, or holes. You never can run Job exactly every Hr 

so You can mix recommendations and check not getCurrentDate() -1hr, but exactly low and high margins

more than 11:00:00 and less than 12:00:00

[Anonymous]

Thanks Vapukov, currently i am going with getCurrentDate() -60mm.

Please suggest how can i specify 11:00:00 and less than 12:00:00. taking consideration that the job runs between the hourly intervals. 

e.g: if the job runs at 11.20 then it should pick the lower limit as 11 and upper limit as 12