File Push/Pull from remote SFTP servers

Anonymous — Mon, 24 Mar 2014 09:11:01 GMT

I have a sub job, which create sftp connections and perform get and put operations. sftp connection details are passed as a delimited string from the main job to sub job.
Eg :
bar|moonshine|bar|bar1234|request|response
maa|moonshine|maa|maa1234|request|response
First column - target identification
Second column - target sftp server
Third column - target user
Fourth column - target user password
Fifth column - download directory (to local)
Sixth column - upload directory (from local)

Sub job will iterate the connection details. if it's a new target it will create the local directory as follows
+ sftp
+-- bar
+ -- request
+ -- response

then it will create corresponding sftp connection and do GET operation from remote request to local request directory.
once it finish it will do PUT operation from local response to remote response directory.
This job is working perfectly for single target. when multiple target presents, i'm getting following exception.
Exception in component tFTPPut_1
2: No such file
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2833)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2185)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2202)
at com.jcraft.jsch.ChannelSftp.stat(ChannelSftp.java:2159)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tFTPPut_1Process(RemoteFileSync.java:1375)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tFTPGet_1Process(RemoteFileSync.java:1182)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tLoop_1Process(RemoteFileSync.java:847)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tJava_1Process(RemoteFileSync.java:506)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.runJobInTOS(RemoteFileSync.java:1693)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.runJob(RemoteFileSync.java:1527)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tRunJob_2Process(ExternalEventsBatchUpdate.java:902)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tJava_2Process(ExternalEventsBatchUpdate.java:786)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate$6.run(ExternalEventsBatchUpdate.java:5170)
Exception in component tRunJob_2
java.lang.RuntimeException: Child job running failed
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tRunJob_2Process(ExternalEventsBatchUpdate.java:922)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tJava_2Process(ExternalEventsBatchUpdate.java:786)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate$6.run(ExternalEventsBatchUpdate.java:5170)

I have tested this job with jcraft 1.5 as well. still gettting the same error.
For your reference i am attaching the sub job screen cast herewith.
Appreciate, a response from anyone.
Thanks n Regards,

Re: File Push/Pull from remote SFTP servers

willm1 — Mon, 24 Mar 2014 13:42:35 GMT

Looking at your first screenshot, I suspect your FTP Get and Put are not synching with your iterate properly - because they're using OnComponentOK - meaning they're not part of the loop subjob. How about placing all the FTP components in a child job (using tRunJob to connect from main) and connect that last iterate to the tRunJob? Then pass any variables to the child job using contexts or global variables. That way, for each loop, you're processing the FTP components in synch for that loop id.

Re: File Push/Pull from remote SFTP servers

Anonymous — Tue, 25 Mar 2014 04:13:17 GMT

Thanks willm, your solution did work

topic Re: File Push/Pull from remote SFTP servers in Talend Studio

File Push/Pull from remote SFTP servers

Re: File Push/Pull from remote SFTP servers

Re: File Push/Pull from remote SFTP servers