Downloading only certain directories from an FTP server

[Anonymous]

Hello Talend Users,

I have a FTP directories with many subdirectories (representing years from 1999 to 2018) . For now I can only download all files in all folders. However I want to only download the folders from 2011 to 2018.

So far my Job looks like this:

How can I iterate over the Folders from 2011 to 2018? With tForeach? And if yes how can I connect it to FTPGet?

Thanks in Advance.

[Anonymous]

Hello,

Do you want to list sub-directories in FTP folder?

Could you please let us know if this resolved topichttps://community.talend.com/t5/Design-and-Development/Exclude-File-Mask-in-TFTPFileList/m-p/124806 is OK with you?

Best regards

Sabrina

[Anonymous]

This is not really the solution to my problem as it is about excluding certain files from the download. I however want to download only the directories 2011 to 2018 in a list of directories which reaches from 1999 to 2018. So somehow I have to provide a list to the FTPget job.

Best regards
Maximilian

The FTP Directory looks like this:

[iamabhishek]

You could use tSSH to filter out the folders using unix commands. The output of tSSH you could then store in global variable which is tSSH_1_STDOUT

Once you have the list of folders in the global variable just use them to iterate over using

tFTPFileList -> iterate ->tFTPGet

Sample unix command to filter out folders starting with "talend" -

"find $(pwd) -maxdepth 1 -type d -name \"talend*\""

[Anonymous]

Hi,

ok sounds really complicated to me since I am a Talend Beginner. How do I connect the tSSH job to the FTPfilelist?

[iamabhishek]

tSSH - > on subjob ok -> tFTPFileList
It might sound complicate but it's not. The solution is not straight forward because FTP support is with files and not with folders - that's the reason for the tweak.