topic Re: Get multiple files with same name with ftp in Talend Studio

Get multiple files with same name with ftp

vpachu — Sat, 16 Nov 2024 04:15:12 GMT

I'm trying to pull files from ftp location and process it. If two files have same name but loaded with different timestamps currently the earlier file is copied to local. How can I copy only the latest and ignore the old.

Re: Get multiple files with same name with ftp

manodwhb — Tue, 29 Oct 2019 00:29:42 GMT

You need to use add the date part to your file name in filemask of tFTPlist.

Re: Get multiple files with same name with ftp

vpachu — Tue, 29 Oct 2019 21:32:19 GMT

The filenames are identical, just the receive timestamps of the files on the ftp server are different. In the below example I want use the second file with timestamp 20191029143000.

Example:

FILENAME | TIMESTAMP

FILE1_20191029163000.ZIP | 20191029140000

FILE1_20191029163000.ZIP | 20191029143000

Re: Get multiple files with same name with ftp

ankit7359 — Wed, 30 Oct 2019 07:22:58 GMT

Hi @vpachu ,

For this scenario, you can add in the checkbox of order by & order action in tfilelist - date modified & Desc -> this setting will enable you to pull in the files which are recently modified. Also you can add in another setting in tFTPGET component -> Below Transfer Mode -> there is overwrite file -> considering the same file name is being sent thru FTP, then you can check the option of overwrite the file.

Pls let me know, in case if something is not clear.

Thanks,

Ankit

Re: Get multiple files with same name with ftp

vpachu — Wed, 30 Oct 2019 14:59:37 GMT

Thanks @ankit7359 . I using this subjob in other jobs to get files. I have overwrite enabled already, but still It's not able to get the latest file, it always ends up with the first file instead of the latest one.