topic Re: delete files more than 45 days (example) old? in Talend Studio

delete files more than 45 days (example) old?

Anonymous — Sat, 16 Nov 2024 14:21:46 GMT

So, I figure once I get a list of these files, I can just use tFileDelete to delete them. (I assume I would feed into tFileList and then make a connection to tFileDelete, like in the pic, but I'm not positive.)

What I don't know is how to read a directory and get a list of the (zip) files that are older that today minus x # of days.

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 22 May 2008 08:23:13 GMT

Hello
The tFileProperties component(under File-->Management family) can fit your need. With tFileProperties component, you can get the create time of the file.
Best regards

shong

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 22 May 2008 18:46:10 GMT

what version is that in? Cause, its not showing up for me in TOS 2.3.3r13892

Ah, ok, I see it for the Perl version, but I'm in Java...

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 22 May 2008 19:18:12 GMT

this bug topic (3545) seems to indicate that the java module is completed in 2.4.0RC1. are the fixes that are in 2.3.3 also in the latest 2.4.0 release? (RC2)

Guess I'm about to find out.

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 22 May 2008 19:41:42 GMT

ok, so... is there somewhere I can find documentation on tFileProperties? cause its not in the most recent documentation available here http://www.talend.com/resources/documentation.php
and its not available in the help linked from TOS : http://127.0.0.1:50949/help/nav/2_2
or can someone tell me how to use this thing? please!

Re: delete files more than 45 days (example) old?

Anonymous — Fri, 23 May 2008 03:53:00 GMT

Hello
The tFileProperties component describes the properties of one file, include:
abs_path: the absolute path
dirname: the direcotry name
basename: the file name
mode_string: read or write
size: the size of file
mtime: the create time of file, return millisecond (long)
mtime_string: the create time of file, return a date (String)
Here is a simple scenario(see the screenshot)
Best regards
shong

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 29 May 2008 20:34:05 GMT

ok... now I can't figure out how I can filter on mtime_string and return either abs_path or basename to tFileList or tFileDelete
I mean, I think I can do some of the filtering in something like tMap, but I don't know if thats the right way to do it

Re: delete files more than 45 days (example) old?

Anonymous — Fri, 30 May 2008 03:37:14 GMT

Hello

ok... now I can't figure out how I can filter on mtime_string and return either abs_path or basename to tFileList or tFileDelete

Here is a Java example, Please see the screenshot.

 /**
     * Returns the number of milliseconds since January 1, 1970, 00:00:00 GMT
     * represented by this <tt>Date</tt> object.
     *
     * @return  the number of milliseconds since January 1, 1970, 00:00:00 GMT
     *          represented by this date.
     */
    public long getTime() {
        return getTimeImpl();
    }

Best regards

shong

Re: delete files more than 45 days (example) old?

Anonymous — Fri, 30 May 2008 20:15:16 GMT

So I tried this process like you said, and it works perfectly... that is, until I try to change the # of days to anything greater than 24
I tried a bunch of different days, and all the ones <=24 worked, and all the ones >24 caused all the files to be deleted
Picture Information
1. contents of the folder before I run the clean up process
2. process with tFilterRow set to delete files created before 25 days
3. results of running the process defined in #2, where all files are deleted (whether they meet the filter criteria or not : before 5/6/08)
2. process with tFilterRow set to delete files created before 24 days
3. results of running the process defined in #4, where only the appropriate files are deleted (any files before 5/7/08)
Note:
tFileOutputDelimited_2 creates out_orig.txt (a list of all the files found in tFileList_3)
tFileOutputDelimited_1 creates out.txt (a list of all the files deleted by tFileDelete_1)
tFileOutputDelimited_3 creates out_notdeleted.txt (a list of all the files rejected by tFilterRow_1)

I find it kind of odd that the advanced filter in tFilterRow_1 works where ### (in the quoted formula below) is <=24, but not greater...

((TalendDate.getCurrentDate().getTime()-(table1.mtime+86400000*###))>0

Re: delete files more than 45 days (example) old?

Anonymous — Fri, 30 May 2008 21:28:24 GMT

Ok, I figured out how to get around the problem I posted above:
1. set ### (in the post above) to a context variable (ex: context.NumDays_Past)
2. set the Type of that context variable to Double
a. You could use Float or Long, in place of Double, here - they both worked for me
b. Don't use Integer or Short, they cause all the files in the folder to delete.
c. Don't use Big Integer, it causes an error (Exception in thread "main" java.lang.Error: Unresolved compilation problem: The operator * is undefined for the argument type(s) int, BigDecimal)

(TalendDate.getCurrentDate().getTime()-(86400000*context.NumDays_Past))>table1.mtime

Re: delete files more than 45 days (example) old?

_AnonymousUser — Tue, 28 Feb 2012 03:42:07 GMT

Hi, I am having error to this context.NumDays_Past
nullpointerexception.
what values you put in context.NumDays_Past?
Regards.

Re: delete files more than 45 days (example) old?

Anonymous — Thu, 09 Jan 2014 10:53:57 GMT

Hi,I have reproduce the problem.
This is cause by 60*60*24*1000*25 out of scope of Integer type(result is: -2134967296)
So the result alway false
Solution:change 60*60*24*1000*25 ---to---> 60*60*24*1000*25L(result is: 2160000000)
Change the result is long type.

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 11:02:10 GMT

Dear All,
I am trying to delete file which are 100 days old , I have used same approach which is mentioned in below email but I AM UNABLE TO UNDERSTAND FOLLOWING THINGS.
STEPS:
1) please check my log files which are in format of: (Name of the JOB_currentdate_time)
Example:==> Log_InsertDIMItems_2014-07-11-13-27-18
This file i have created on Friday 11th July
I want to delete files which are older than 100 days.
2) Screenshot of my job:
3) Code in tFilter Component: Advance Code:
(TalendDate.getCurrentDate().getTime()-(File_prop.mtime+86400000*24))>0
By using this query(24 days limit) then 0 files are deleted , but by using 25 all files are getting deleted which is confusing for me. Please advise where I am wrong ?
I am not getting how this is happening.
Can anyone please guide me on same.
My requirement is of deleting files which are 1oo days old .
Thanks and Regards
RSH

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 11:08:16 GMT

Hi RSH,
tFileList-->tFileProperties-->tJavaRow-->tFileDelete
Use this flow, assign file names to context variables in tJavaRow based on your date condition and use that context variable in tFileDelete component for file to delete..
try this
vaibhav

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 11:13:59 GMT

Hi vaibhav,
Thank you for reply.
Its not possible to declare context for each file because we have near about 30 different names of files. Is there any possibility to manipulate on current snapshot which i provided?
I am not getting where I made mistake ?
Regards
RSH

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 11:24:36 GMT

If I use :
(TalendDate.getCurrentDate().getTime()-(File_prop.mtime+86400000*25))>0
All files are getting DELETED (--NOT SURE HOW)
Please find below is an snapshot of result:
mtime mtime_string GetDate Calculated_Data_OF_Advance_Filter
1,40508E+12 Fri Jul 11 13:27:20 CEST 2014 1,40533E+12 2390162346
1,40508E+12 Fri Jul 11 13:27:20 CEST 2014 1,40533E+12 2390162867
1,40508E+12 Fri Jul 11 13:27:19 CEST 2014 1,40533E+12 2390163079
1,40508E+12 Fri Jul 11 13:27:20 CEST 2014 1,40533E+12 2390162220
1,40508E+12 Fri Jul 11 13:27:20 CEST 2014 1,40533E+12 2390162663

If i use
(TalendDate.getCurrentDate().getTime()-(File_prop.mtime+86400000*24))>0
NO file is deleted..

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 11:40:29 GMT

Hi,
mtime is unix timestamp in millisecond.. 86400000 represents one day
TalendDate.getCurrentDate().getTime()-86400000*2) - (input_row.mtime)>0
Above code deletes file which are older than two days.. you replace 2 by 100, then it should do the job... use wherever required in your design...
Try this.
vaibhav

Re: delete files more than 45 days (example) old?

Anonymous — Mon, 14 Jul 2014 12:06:47 GMT

Hi Vaibhav,
I used the query which you suggested, but still strange result.
All files from server got deleted
Not sure how this is working out:
Query:
(TalendDate.getCurrentDate().getTime()-86400000*100) - (File_prop.mtime)>0

Re: delete files more than 45 days (example) old?

Anonymous — Tue, 15 Jul 2014 05:04:28 GMT

Hello ,
Can anyone please guide me on my issue please ?
Thanks in advance.
Regards
RSH

Re: delete files more than 45 days (example) old?

Anonymous — Tue, 15 Jul 2014 06:43:08 GMT

can you try below code. it will help you on this issue.
use this code in If condition.

new Date(File_prop.mtime).before(TalendDate.addDate(TalendDate.getCurrentDate(), -44, "dd"))==true

you can convert File_prop.mtime date into Date format using new Date(File_prop.mtime) constructor then you can add or minus days from date and compare with current date .