topic Re: Load new files by names from FTP in Talend Studio

Load new files by names from FTP

_AnonymousUser — Thu, 22 Oct 2015 10:59:39 GMT

Hi, everyone!
Please, help me to solve the problem.
Everyday on FTP server is new files. Quantity of new files can be different. It's nessesary to get new files, read them and write data to differrent tables accoding to filemask. I have table, wich contain filenames of loaded files.
My idea is to get FTP filelist with given mask, get filelist from DB, compare, and using FTPGet load new files.
I use:
tFTPFileList -->tFTPFileProperties -->tMap<--tMysqlInput
|
tJavaRow(Put filename into Global = globalMap.put("DOC_NAME", doc_name))
|
tFTPGet(filemask: (String)globalMap.get("DOC_NAME"))
But the last step doesn't work
Can you help me with that please

Re: Load new files by names from FTP

Anonymous — Thu, 22 Oct 2015 16:24:09 GMT

Hi
The initialization of file mask of tFTPGet is executed before tJavaRow with this job design, that's why you get nul value on tFTPGet, try to change your job to:
tFTPFileList -->tFTPFileProperties -->tMap<--tMysqlInput- tJavaRow-->oncomponentok--tFTPGet

Best regards
Shong

Re: Load new files by names from FTP

_AnonymousUser — Fri, 23 Oct 2015 10:03:04 GMT

Thank you, Shong!
OnComponentOk lets get remote files. But comparison is too long (more than 1000 files are compared everytime).
I decided to put the result of comarison into tHashOutput, and onSubjobOk
tHashInput
|
tJavaRow(Put filename into Global = globalMap.put("DOC_NAME", doc_name))
|
OnComponentOk
|
tFTPGet(filemask: (String)globalMap.get("DOC_NAME"))
But in this case tFTPGet loads only one file, and tHashInput contains five files.
What should I do in this situation.
Thanks!