Move (cut) files instead copy ftp to local folder

[JavierP2]

Hi there,

I'm working on a program that moves files between a SFTP server and a local folder.

It works fine but I have a problem, seems as the tFTPGet copy the files from the SFTP and move them to the local folder but the files still remain into the SFTP. I do need to cut the files instead copy, is there any way?

I've tried with a tFTPDelete but this solution doens't work 100% as is for example during the process between the FTPGet and the tFTPDelete a new file comes it auto deletes as is on the SFTP and it's not copied.

So solutions that I think that can work:

1) Cut the files instead copy 

2) Any rule into the tFTPDelete to only delete the ones listed into the tFTPFileList

Any ideas?

Thanks a lot!

[billimmer]

then shouldn't there be an iterate connecting the tFTPFileList to a tFTPGet.  And let it iterate over your files so you don't need 4 gets?

[JavierP2]

I will try to explain the full procedure, so maybe it can makes sense to you and maybe there's an easier way to do it (it's my 3 time with talend).

So I have SFTP Folder where I will receive many files during the day. 

With this files I want to run every "x" mins this program that have to do the following:

1) Copy the files from the SFTP that start with "XX" into 1 local folder

2) Copy the files from the SFTP that start's with "BB" into other local folder

3) Once copied to the 2 folders I want to rename these files from the SFTP (with "OK_+FILENAME") and move them to 2 different "Historic" local folders. (that's why there's 4 gets) cause I have 4 different names and folders to move.

4) Remove the files from the SFTP.

5) Send a confirmation mail (this works).

[billimmer]

That all makes sense.  

First rename everything "OK_" + so you can keep track of what you are doing

tFTPFileList_1----(iterate)---->tFTPRename---(OnComponentOK)----tFTPDelete

Now you are only working on "OK_" files and don't need to track new stuff.  Next build your email list for only files that start with "OK_", just like you are already doing, but set the tFTPFileList_2 to "OK_*"

tFTPFileList_2----(iterate)---->tIterateToFlow---(main)----tLogRow---(main)--->tJavaFlex

Now you are ready to do work  Iterate through the "OK_XX*" files and do your work.  Use a filemask in the tFTPFileList so you only get these files.  copy them to local, archive to history folder, and delete each one.

tFTPFileList_3----(iterate)---->tFTPGet---(OnComponentOK)---->tFTPRename---(OnComponentOK)---->tFTPDelete

Now do the same for the "OK_BB" files

Finally, send your email

[JavierP2]

Thanks for the reply, youre helping me a lot.

I need to change 1 thing, for the first movement, the files cannot have the OK_ added, So OK_ it's only for the files that will go to the historical folder.

It makes any change on your idea? I will try to replicate.

Thanks again!

[billimmer]

You can strip off the "OK_" after you copy it to local.

Or instead of adding "OK_", you can move the files to a "OK" folder, and then do all your work from there.

[JavierP2]

Hi,

I've managed to do it with the renaming at the begining, I think this way there's less time between jobs so it would be more 'safe'.

I have a last question, it seems as work but in any way I need to close the connection with the FTP and also send the email.

So I want in any way to connect the tFTPDelete_2 to the tFTPClose when there's no more files to process and also send the email. Not sure how to connect.

I've thought about an if into the tFTPDelete Like : ((Integer)globalMap.get("tFTPFileList_4_NB_FILE"))==0 

But it's not working.

Any ideas how to connect that?

[billimmer · Accepted solution]

you should connect each subjob.  So connect

tFTPFileList_1 to tFTPFileList_2 with a OnSubjobOK

tFTPFileList_2 to tFTPFileList_3 with a OnSubjobOK

tFTPFileList_3 to tFTPFileList_4 with a OnSubjobOK

tFTPFileList_4 to tFTPClose_1 with a OnSubjobOK

[JavierP2]

Worked like a charm.

Thanks a lot, you saved my day!!