topic Re: tFileFetch to get multiple XML files in multiple directories in Talend Studio

tFileFetch to get multiple XML files in multiple directories

_AnonymousUser — Thu, 25 Oct 2012 00:35:08 GMT

Im a novice to Talend but I'm starting to learn some things. I have a job setup to read XML files on my server and load them, what I need help with is pulling the files down from the web and storing them on my machine.
The XML files are in this directory structure:
http://www.banking/2012/09/01/bank.xml
http://www.banking/2012/09/01/banks/1001.xml
http://www.banking/2012/09/01/banks/1002.xml
http://www.banking/2012/09/02/bank.xml
http://www.banking/2012/09/01/banks/3452.xml
I think you get the idea.
I think I can use a tForEach to help do this but I don't know exactly how to set this up. Some of the XML file names are static while others are dynamic so that is another challenge. Any experts out there able to help out?
Thanks gang!

Re: tFileFetch to get multiple XML files in multiple directories

_AnonymousUser — Thu, 15 Nov 2012 02:57:33 GMT

Does anyone have some thoughts on this problem? Thanks!

Re: tFileFetch to get multiple XML files in multiple directories

Anonymous — Thu, 15 Nov 2012 03:32:20 GMT

Hi
tFileFetch don't support a dynamic file name in URI field. You can consider to use FTP or SCP protocol to get the files.

Re: tFileFetch to get multiple XML files in multiple directories

Anonymous — Sat, 17 Nov 2012 17:00:07 GMT

Thanks for the reply. I have tried to FTP to the website but that doesn't appear to work. I'm fairly certain the site is not setup to accept FTP Transfers. I'm going to look into SCP and see if that is successful.