Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

File Push/Pull from remote SFTP servers

I have a sub job, which create sftp connections and perform get and put operations. sftp connection details are passed as a delimited string from the main job to sub job.
Eg :
bar|moonshine|bar|bar1234|request|response
maa|moonshine|maa|maa1234|request|response
First column - target identification
Second column - target sftp server
Third column - target user
Fourth column - target user password
Fifth column - download directory (to local)
Sixth column - upload directory (from local)

Sub job will iterate the connection details. if it's a new target it will create the local directory as follows
+ sftp
+-- bar
+ -- request
+ -- response

then it will create corresponding sftp connection and do GET operation from remote request to local request directory.
once it finish it will do PUT operation from local response to remote response directory.
This job is working perfectly for single target. when multiple target presents, i'm getting following exception.
Exception in component tFTPPut_1
2: No such file
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2833)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2185)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2202)
at com.jcraft.jsch.ChannelSftp.stat(ChannelSftp.java:2159)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tFTPPut_1Process(RemoteFileSync.java:1375)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tFTPGet_1Process(RemoteFileSync.java:1182)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tLoop_1Process(RemoteFileSync.java:847)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.tJava_1Process(RemoteFileSync.java:506)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.runJobInTOS(RemoteFileSync.java:1693)
at externalbatchupdate.remotefilesync_0_1.RemoteFileSync.runJob(RemoteFileSync.java:1527)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tRunJob_2Process(ExternalEventsBatchUpdate.java:902)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tJava_2Process(ExternalEventsBatchUpdate.java:786)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate$6.run(ExternalEventsBatchUpdate.java:5170)
Exception in component tRunJob_2
java.lang.RuntimeException: Child job running failed
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tRunJob_2Process(ExternalEventsBatchUpdate.java:922)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate.tJava_2Process(ExternalEventsBatchUpdate.java:786)
at externalbatchupdate.externaleventsbatchupdate_0_1.ExternalEventsBatchUpdate$6.run(ExternalEventsBatchUpdate.java:5170)

I have tested this job with jcraft 1.5 as well. still gettting the same error.
For your reference i am attaching the sub job screen cast herewith.
Appreciate, a response from anyone.
Thanks n Regards,
0683p000009MDMf.png 0683p000009MD8j.png 0683p000009MDJh.png 0683p000009MDMk.png 0683p000009MDMp.png 0683p000009MDIj.png 0683p000009MDMu.png
Labels (3)
2 Replies
willm1
Creator
Creator

Looking at your first screenshot, I suspect your FTP Get and Put are not synching with your iterate properly - because they're using OnComponentOK - meaning they're not part of the loop subjob. How about placing all the FTP components in a child job (using tRunJob to connect from main) and connect that last iterate to the tRunJob? Then pass any variables to the child job using contexts or global variables. That way, for each loop, you're processing the FTP components in synch for that loop id.
Anonymous
Not applicable
Author

Thanks willm, your solution did work