SFTP multiple files using tFTPPut

[iamabhishek]

Trying to build a solution to transfer files to an FTP server from local on-premise system. Currently the process is only transferring the last file of the lot found from the directory. Tried debugging the job but couldn't understand where the actual problem is. 

What I have tried so far:

Used tFileList to get the list of files for a given directory, given "*.txt" as FileMask.

Used tIterateToFlow to iterate through the list of files. The Schema is of a single column "fileName" with the value set as - ((String)globalMap.get("tFileList_1_CURRENT_FILE"))

Finally used tFTPPut to SFTP the files to remote directory. Local Directory is set as - ((String)globalMap.get("tFileList_1_CURRENT_FILEDIRECTORY")) & FileMask as - ((String)globalMap.get("tFileList_1_CURRENT_FILE")). Overwrite is the option chosen.

Screenshot of the job -

Thanks in advance for any kind of help and let me know if more information is required for this.

Platform: Talend Open Studio for Data Integration Version: 6.5.1

[manodwhb]

@iamabhishek,you no need to use tIteratetoFlow,since you can directly connect Iterate flow from tFilelist to tFTPPut and in between you cna use tJava componet to print the current file name.

[iamabhishek]

@manodwhb Have tried that too. If we don't use tIterateFlow and directly pass the iteration from tFileList to tFTPPut then only the first file would be retrieved and sent and for the second file there would be en error - "No File Present".

This is the output when debugged using tJava - 

[statistics] connecting to socket on port 3583
[statistics] connected
[trace] connecting to socket on port 5031
[trace] connected
Output.txt
C:/FTP/dir
Sample.txt
C:/FTP/dir
No File Present
[statistics] disconnected
[trace] disconnected

[manodwhb]

@iamabhishek,i have used below way and working for me.check your desing based on this.

[manodwhb]

@iamabhishek,below settings also working as expected.

[iamabhishek]

@manodwhb Thank you for the solution. 

Noticed that you have used "*.txt" as FileMask in the tFTPPut which ensures that it will pull all the files from the CURRENT_FILEDIRECTORY. In that case we don't even need to iterate the files in the directory and we don't need tFileList too.

But the desired approach should have been like get the files using tFileList and iterate over the list and using tFTPPut transfer the files. The FileMask should be used in tFileList and whatever the filename is passed in that iteration tFTPPut should use that preferably using - CURRENT_FILE  

[manodwhb]

@iamabhishek,you can find my other sceernshot i have used ((String)globalMap.get("tFileList_1_CURRENT_FILE")) as file mask in tFTPPut and working fine.

[manodwhb · Accepted solution]

@iamabhishek

[iamabhishek]

Thank you @manodwhb, sorry I might have overlooked your reply.

Yes, your solution is working perfectly for the list of files in local directory much appreciated.

One thing I noticed here is and the reason for my process not working was -

I was using tFTPPut --> (on component ok) --> tFTPClose , wherein you have used

tFileList --> (on subjob ok) --> tFTPClose which has clicked. 

Any idea on this.

[manodwhb]

@iamabhishek,since you need to design job as i shared in screenshot, other wise in your design first file will be placed and for second file tFTPPUT will fail that connection alreday closed.that you have used on component ok of tFTPPUt to tFTPCLOSE