Error and missing file while downloading using SFTP

[ars]

Hi all,

I've developed a job in order to download files from an SFTP server. It's being up and running for many years and I didn't have any problem in the past.

A special notice to consider, is that the SFTP server is hosted on external partner prremises and according to its configuration, files are being deleted after the downloading is completed.

So, everything work fine so far but recently I'm facing an unknown (and strange!) issue.

Under unspecified circumstances, component tFTPGet throws the error message {com.jcraft.jsch.SftpException} and the job is terminated.

But while the file has not been uploaded (not even partially) the SFTP server considers that was downloaded and the file is removed from it. So, if I try to re-download it, it's not there!!!

Any thoughts, why the Talend could crash and the file is being lost??

Best regards,

Ars

[Anonymous]

Hello,

By design, tFTPGet will look at your local drive (i.e. C drive, C:/ASDG/DEX) to place/store your file , so are you using local directory instead of remote directory in it?

Could you please post the full error stack trace here and send us a snapshot of what you are seeing or trying to do so we can better understand the issue.

And then I will look into your issue as soon as possible and get back to you.

Best regards

Sabrina

[ars]

Hello,

Thank you for your reply.

Basically, my tFTPGet component uses a local network drive, to store the downloaded files:

 

Regarding the full error stack trace, unfortunetally the only error I can capture is through log (tLogCatcher info).

This is because the job is executed automatically via central pc and at the moment (when) it fails I have no view of the console.

Also, I can't reproduce the problem 'ad hoc' because the SFTP where we download the files is on an external partner and I have no rights to upload a file. But mostly, even I had the right to do it, it might be impossible finally to reproduce the error. Because, the exception occurs some times and this is what we need to find. The reason of why this is happening.

This is what tLogCather, exports on log table:

There is no any other 'hidden' information on the field. Only this value: com.jcraft.jsch.SftpException:

Any thoughts?

BR,

Ars

[Anonymous]

Hello,

We can see that you are using a relative path as your local directory. There is access to this shared network drive? And It does work well when get files from FTP server?

In addition, if you don't mind, could you please open the error log generated automatically by the Studio to see whether you have error stack trace there; if you do, we will appreciate it a lot if you can paste it here.

About how to open this error log: in your Studio, click Window to open the menu, then select Show View->General->Error Log. Once you see an error, double click it.

Best regards

Sabrina

[ars]

Hello,

Yes, job has full access on the shared folder and it does work well when we get files from the SFTP server.

Now, regarding the log you need, as I mentioned the job is executed automatically via central pc.Thus, at the moment of execution there no any Studio instance open and only the .bat (from the deployment package) file is executed.

And I can't reproduce the error from my Studio instance; this is something that is happening radomly from the automated job.

Is there any other log info available from the 'runtime' job?

BR,

Ars

[Anonymous]

Hello,

Do you mean this FTP job can be executed successfully in studio? What's your runtime server for .bat file?CMD? Window scheduler?

Best regards

Sabrina

[ars]

Hello,

The FTP job executed succefully in studio. On runtime server it's executed (most times succefully) using the .bat file, which is triggered via Window scheduler.

BR,

Ars

[Anonymous]

Hello,

As this job can be executed successfully in talend studio, I'm supposing something be wrong with Windows task scheduler for using the mapped disk drive letters.

Have you tried to add some debugging code in your job .bat file to see if there is any issue?

Best regards

Sabrina

[ars]

Hello,

The windows scheduler is working well. The job is executed on daily basis and most times it's completed successfully.

So this is not the point, of our investigation! The question is why the tFTPGET fails in some cases and raises the SFTP exception.

BR,

Ars

[Anonymous]

Hello,

Do you have any record when tFTPGET fails? What's kind of some cases? Specific environment cases?

Is there any possible or detailed steps for us to repro this issue?

Best regards

Sabrina