topic Re: Get Files from FTP server to Local Machine in Talend Studio

Get Files from FTP server to Local Machine

Anonymous — Mon, 26 Aug 2013 08:58:50 GMT

Dear all,
I got struck in one issue which is as follows:
Requirement:
1) Get text file from FTP server to Local Machine, if we get files then rename this file by yyyyMMdd+name of the file+OK.text
2) We need to delete those text file from local folder which are 1 month OLD.
Development:
1) I have used Tftpget component to get files from FTP server to Local Machine, but i am unable to RENAME it LIKE yyyyMMdd+name of the file+OK.text becase i am not able to provide path of this file(Local Machine path).If i give remote directory path in tftprename component then it work but it rename in FTP server which i dont want.i want to make change of name in LOCAL machine only.

2) Not yet done.
Can you please help me with step by step process ....pleaseeee...
Thanks & Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Mon, 26 Aug 2013 09:58:46 GMT

Hi,

Get text file from FTP server to Local Machine, if we get files then rename this file by yyyyMMdd name of the file OK.text

Actually, there is no component to just rename a file on your local machine. You can either use tFileCopy and check "Remove source file" or use tSystem to call a system command or tJava with your own code.

We need to delete those text file from local folder which are 1 month OLD.

If the source file name has yyyyMMdd name(such as: 20130725xxOK.txt), you can use a specified file mask in tFileList to get the 1 month OLD source file and then delete them by tFileDelete.
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Mon, 26 Aug 2013 12:38:52 GMT

Hello Sabrina ,
Can you please guide me what code to write in tsystem to change file name?
My file path is "D:/Talend/Project/Finacle/_Logs/Log.txt"
I want to rename this file (Log.txt) to "yyyyMMDD+Log+OK.txt"
Or is there any easy way to do it?
Thanks & Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 05:37:47 GMT

Hello Sabrina & Team,
Good Morning !!!
I am trying to change file name by using tsystem component with following command:
Files path is : "D:/Talend/Project/Finacle/_Logs/Log.txt" Reguired name of file is : Changed_Log_File.txt
in Tsystem i am using folowing command :
"cmd /c REN D:/Talend/Project/Finacle/_Logs/Log.txt Changed_Log_File.txt
I am getting an error of incorrect syntax. Can you please guide me how should i RENAME my old file with new name.
Thanks in Advance.
Regards
RSH.

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 05:57:23 GMT

Hi,
You can use use tFileCopy and check "Remove source file"(if you don't want the source file).
To make it clear, I have designed a demo job: tfilelist-->iterate-->tfilecopy
See my screenshot for details
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 07:14:37 GMT

Hello Sabrina,
Thanks for your guidance...
I made the job just like you but getting errors like:
Exception in component tFileCopy_1
java.lang.RuntimeException: The source File "/home/FTP_Server/LOGS_Folder/LOG.txt" does not exist or is not a file.
My requirement is to Get file from FTP server to Local Machine and is it is successful then RENAME this file,
so is tfilecopy component Copy files from FTP server to Local machine?
Please advice me because i am running out of time for deployment.
Thanks & Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 07:28:37 GMT

Hi,

Exception in component tFileCopy_1
java.lang.RuntimeException: The source File "/home/FTP_Server/LOGS_Folder/LOG.txt" does not exist or is not a file. My requirement is to Get file from FTP server to Local Machine and is it is successful then RENAME this file,
so is tfilecopy component Copy files from FTP server to Local machine?
Please advice me because i am running out of time for deployment.

The tFileCopy component cannot copy a file from FTP to Local machine. You should get the file from FTP to local machine firstly and then use tFileCopy to rename them.(If you don't want to use tFTPrename to rename files in FTP).
That means: tFTPConnection-->onSubjobOK-->tFTPget-->onSubjobOk(onComponentOk)-->tfilelist-->iterate-->tfilecopy.
In tFTPget, the files has been got from FTP into Local Machine where you can use tfilecopy to copy and rename.
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 08:32:06 GMT

Hello Sabrina,
I tried to work in my job as per your guidance and i got SUCCESSFUL Run..\
Thanks you Sabrina ..
Some Snapshots i am attaching for all other developers...
Now i just remain with DELETE those files from Log folder which are 1 month OLD.
Thanks & Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 08:36:02 GMT

Hi,

Now i just remain with DELETE those files from Log folder which are 1 month OLD.

What the file name looks like?yyyyMMDD+Log+OK.txt?

If the source file name has yyyyMMdd+name(such as: 20130725xxOK.txt), you can use a specified file mask in tFileList to get the 1 month OLD source file and then delete them by tFileDelete.

Is it working for you?
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 08:39:27 GMT

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 08:51:50 GMT

Hello Sabrina,
I am unable to attach snapshots ? not getting reason of attachment failure???
My log folder contains Files like 20130101_log_OK.txt it is in YYYYMMDD format.
Assume today is 1st of Feb 2013( 20130201) then my log folder should now have files prior to 1st jan 2013 (20130101)
Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 10:09:43 GMT

Hi,

I am unable to attach snapshots ? not getting reason of attachment failure???

For registered users, clicking on "post reply" (not through the quick post") and pay close attention on Limitation: 20 images per post, each image must be less then 1024x768 pixels and 200 KB, .png is preferred.
Are you looking for that?
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Tue, 27 Aug 2013 10:36:04 GMT

Hello Sabrina,
Actually process is i have an folder called LOG_Folder in which every day 1 text file will be added by talend Job scheduler.
Now requirement is i have to DELETE text file which will be greater than one month OLD.
Example:
Today is 27/08/2013 so today my job will be run and i will have 1 log text file in log folder, but after adding this file i have to delete those text files which are old more than 1 months i.e. Files Before 27/07/2013.
Please guide me or provide me steps(snapshots) so i can proceed further..
Thanks & Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Wed, 28 Aug 2013 08:57:26 GMT

Hi,
Sorry for delay!
You can design a dynamic file mask in tFileList to get the last month logs and then delete them together(I mean files belong to July but not Files Before27/07/2013, such as,31 files will be deleted which include July in their file name).
The job work flow should be:tjava-->onSubjobOk-->tFileList-->tFileDelete
The code part:

         
java.util.Calendar calendar = java.util.Calendar.getInstance(java.util.TimeZone.getDefault(),java.util.Locale.ENGLISH);
        calendar.add(calendar.MONTH, -1);
        java.text.SimpleDateFormat dateFormat = new java.text.SimpleDateFormat("yyyyMM"); //$NON-NLS-1$
        dateFormat.format(calendar.getTime());  
        
        globalMap.put("key", dateFormat.format(calendar.getTime()));   
        System.out.println(dateFormat.format(calendar.getTime()));

See my screenshots for details
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Wed, 28 Aug 2013 10:41:54 GMT

Hello Sabrina,
Thankful for your efforts to help me.
Just like to inform you, i do have 7 text files in my LOG folder
1) 20130727_log.txt
2) 20130728_log.txt
3) 20130729_log.txt
4) 20130730_log.txt
5) 20130731_log.txt
6) 20130826_log.txt
7) 20130827_log.txt
😎 20130828_log.txt
Our today date will be generated by Talenddate.getdate(yyyymmdd) and it will be stored in context.
(Suppose today is 20130828)
So if i run the job it should delete only 2 files like
1) 20130727_log.txt
2) 20130728_log.txt
because ( Systemdate - 1 month ) = 20130728 ,
but presently when i Run our job it DELETE following files:
1) 20130727_log.txt
2) 20130728_log.txt
3) 20130729_log.txt
4) 20130730_log.txt
5) 20130731_log.txt WHICH IS WRONG...
Please suggest what changes need to be done in CODE?
Can you please tell me what is this code: ((String)globalMap.get("key")) ?? i dint got by "ctrl+ Space"
Thanks in Advance...
Regards
RSH

Re: Get Files from FTP server to Local Machine

Anonymous — Wed, 28 Aug 2013 10:55:46 GMT

Hi,

Can you please tell me what is this code: ((String)globalMap.get("key")) ?? i dint got by "ctrl+ Space"

It is "Global variables". From the code in java, you can get a date"201307".
I put the date into a "Global variables"((String)globalMap.put("key",date)) so that tFilelist can use "((String)globalMap.get("key",date)))" in file mask.
NOTE:"key" is just the name of Global variables and it is changeable.

So if i run the job it should delete only 2 files like
1) 20130727_log.txt
2) 20130728_log.txt
because ( Systemdate - 1 month ) = 20130728 ,
but presently when i Run our job it DELETE following files:
1) 20130727_log.txt
2) 20130728_log.txt
3) 20130729_log.txt
4) 20130730_log.txt
5) 20130731_log.txt WHICH IS WRONG...

It seems that you need a date field "20130701-20130728", I need make a investigation on it and will come back to you asap.
Best regards
Sabrina

Re: Get Files from FTP server to Local Machine

Anonymous — Mon, 09 Sep 2013 14:06:11 GMT