Get two latest files from ftp directory

[RA6]

Hello folks,

I am trying to retrieve two last files below [different names] from an ftp directory.

Expected result :

Lastest File 1 : Clients_26102018.csv

Lastest File 2 : Ecritures_26102015.csv

Based on the tftpfileproperties, i have filtered the mtime by desc and fetching the first row and i am retrieving my first file => Clients_26102018.csv

As you can see, i am using filemask "Client_*.csv" to retrieve the file.

Apart from duplicating the same job for the other file, I want to know if there is another way to achieve this.

something like foreach parameters? The only thing thing that is changing  is the file mask.

Or is there way to retrieve both files in one job?

Thank you.

Best regards,

Ashish

[Anonymous]

Hi Ashish,

      Why don't you fetch all the file property details to buffer by giving the file mask as "*"? Once you get the file details, you can group the files to different groups based on file names (you will need some regex/substring function here based on your use case) and pick the file with oldest date.

      Once the required file is identified in each group, you can do file fetch for only those files.

Warm Regards,

Nikhil Thampi

[RA6]

@nthampi

Thank you for your response.
I am new to talend. Can you show me how to group the files to different groups based on file names please?

Best regards,
Asadasing

[Anonymous · Accepted solution]

Hi,

     First of all my sincere apologies for responding back. I am currently on vacation but thought to update this post as per my promise.

     I have created the current flow using tFIlelist in local but you can create the same logic for FTP flow also using tFTP components.

     Below are the sample files I have created and my intention is to pick the latest files(highlighted) based on file creation timestamp.

 The job flow below is picking the files and the last subjob can be used (instead of tfileproperties_2, you can use other components) to process them further.

I am attaching the job for your reference. It is quite self explanatory. The only change you need to do is to use the FTP components for file list and reading.

If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

child.zip

[RA6]

Genius!!
Thank you @nthampi

Best regards,
asadasing

[Anonymous]

my version is 6.5.1