topic Re: FTP problem. Getting 0KB files in Talend Studio

FTP problem. Getting 0KB files

Tech8 — Sat, 16 Nov 2024 07:47:59 GMT

I have a simple job to get XML files from a sFTP server to a local directory. My job is working perfectly but sometimes im getting empty files and i cant figure out whats the problem. The job is something like tFTPConnection -onComponentOK-> tFTPFileList -iterate-> tFTPGet. I close the connection with an onSubjobOk from the tFTPFileList. In my Connection i have to use the SFTP Support.

Did somebody have the same problem or know how this is happening?

Re: FTP problem. Getting 0KB files

fdenis — Mon, 20 Aug 2018 09:09:46 GMT

your file may be empty when you get it for the first time! (it happen when you create and fill file into the ftp folder).
Solution 1 is to have a ready flag file who is add to ftp folder when file is generated.
Solution 2 is to download file 2 time (with delay) to check than the size is the same.

Regards

Re: FTP problem. Getting 0KB files

Tech8 — Mon, 20 Aug 2018 09:39:10 GMT

Can you explain a little bit more the first solution? I dont really know if this will help me but just in case.

The 2nd one is not possible for me because when the i download the file it gets deleted from the SFTP server so cant download it again. The thing is that the file in the server has size (5KB) but when i get it to my local directory, the file is empty (0KB).

Thanks so much for your reply.

Re: FTP problem. Getting 0KB files

fdenis — Mon, 20 Aug 2018 10:35:40 GMT

when you build a file: you create the address into the allocation table then you fill datas int the file.
sometime when file is created on an ftp folder, you can download file between this steps so you download an empty file.
the best way is to create file in a work folder, then copy it into ftp folder.

Solution 1: ask the file builder to add a semaphore. it's an empty file who say file is ready to be download. when the data file is completely present into ftp folder create an file "nameOfDataFile.ok".

Solution 2: is used when you cannot add semaphore. it's a simple wait (how many time do you need to fill this file?).

good luck.

Re: FTP problem. Getting 0KB files

Tech8 — Tue, 21 Aug 2018 08:32:48 GMT

I just need to fill each file one time. So if i add a wait when im doing the tFTPGet will give the server time to fill the file completly?

Re: FTP problem. Getting 0KB files

fdenis — Tue, 21 Aug 2018 08:50:11 GMT

a wait is just a delay. You can get list of file to download after the download job. if you run your job each 5 min your delay will be 5 min…

get the list --- wait --- download ------get the list --- wait --- download ------…..
regards,