read multiple csv filenames with timestamp

[Anonymous]

Hi,

I'm new to talend. I'm trying to work on a job for below scenario.

1.Daily multiple csv files with filename having timestamp are loaded in ftp folder.

2. I need to read each csv file and work and load it into separate table in mysql.

Example of filenames:

AAA_21042020_144530.csv

BBB_21042020_144535.csv

CCC_21042020_144538.csv

These three files have to be read using separate tfileInputDelimited option and load into respective tables.

But as the filenames have timestamp which changes daily, we cannot have wildcard option in tfileInput Delimited.

After exploring I found, we can use tfileList to iterate through multiple files and use filemask. But to read multiple files with different patterns , multiple tfilelist have to be opened.

Can someone share a better solution to this problem.

[TRF · Accepted solution]

1 single tFileList connected to an empty tJava which is in turn connected by 3 if triggers to 3 tFileInputDelimited.
If conditions are based on tFileList_1_CURRENT_FILE to determine which way to go using String.contains() or String.startsWith() methods.

[Anonymous]

Thank you TRF for sharing the solution.

I would like to know, how do I fetch files from multiple folders into tFileList and carry out the remaining steps.

Scenario:

1. I have files placed in multiple folders, which i need to read through tFileList.

2. Then based on filename, I need to load data into corresponding table.

For point 2, the solution you provided is working. I need inputs for point 1.

Thanks.

[martel]

Why dont you use the tFileList with regex : _(21042020)_ 

and have the list only for the day 21042020  (current_date day ?)

and for each in the list order by the second party (_144530.csv) ascending  call your open file csv

[Anonymous]

Hi,

I need solution to read files from different folders and also pick the latest files, if there are mutliple copies of files placed in a folder.