topic Re: processing files with tFTP in Talend Studio

processing files with tFTP

_AnonymousUser — Wed, 08 Aug 2012 15:09:15 GMT

I have a ftp site that I need to fetch files from on a weekly basis. the file name is of the form trYYYYMMDD.zip, where YYYYMMDD is the date of their data. they will keep old files out there for archive purposes, so how do I get the latest zip file? I have a tFTPConnection to a tFTPFileList to a tFTPGet that works fine if I hard code the name, but i cant find anything to get the latest file. so can anyone help? (i am just learning java also)

Re: processing files with tFTP

Anonymous — Wed, 08 Aug 2012 15:39:09 GMT

Hi,
What I would do is the following:
1. Get all your filenames
2. Sort them by using tSortRow (sort descending, so the last file is the first row)
3. tSamplerow: Only keep the first row, by setting the range to 1. ( so you only keep the last filename, which you can then use to get the file)
4. Read the file
Regards

Re: processing files with tFTP

_AnonymousUser — Wed, 08 Aug 2012 17:12:02 GMT

Thanks for the suggestion. I tried adding this (tFTPCOnnection -> tFTPFileList -> tSortRow -> tSampleRow -> tFTPGet). TFTPFileList will only iterate to the tSort and I do not have a "sync columns" in my tSortRow as the doc alleges, also tSort is a "_4" where all other components are "_1" if this means anything.
It wont compile; tSample Row has multiple errors: "IterateStruct cannot be resolved to a type" multiple times and "filename cannot be resolved or is not a field" multiple times and a warning for the tSort "this component should have input link(s)". the filename issue i think is the tSort, where i set a column called filename to sort since I had no "sync columns" to connect to the previous.
I think I need something between the tFTPFileList and the tSortRow

Re: processing files with tFTP

alevy — Thu, 09 Aug 2012 14:58:10 GMT

You need to add a tIterateToFlow between tFTPFileList and tSortRow, where you can create a flow containing the file names.
The numeric postcript for the component's unique name is irrelevant.

Re: processing files with tFTP

_AnonymousUser — Thu, 09 Aug 2012 16:11:52 GMT

mpa and alevy thank you both. It now works exactly as I need it too. you have helped and taught me a lot. Now I can continue on with the rest of the processing. Again, thanks for the help.