How to get latest files

[Anonymous]

How to get the latest file. when we run history. Can anyone please help to sort it out.

[TRF]

Have a look to tFileList component options to sort files based on chronological order with older first.

[Anonymous]

There is only modified in tfilelist. 

In my scenario I'm downloading all the history files in job server, but based on filedate it should process only latest date file.

Eg: 1.Customersdata_20180126.psv

       2.Customersdata_20180201.psv

       3.Customersdata_20171231.psv

We have to process only  2.Customersdata_20180201.psv file which is latest

Thanks,

Venkat

[TRF]

Same answer based on filename.
Memorize the 1rst name using a global variable then reuse the variable in the next subjob to get the file.

[Anonymous]

Can you please explain in detail, because file names may vary.

[manodwhb]

first put in the memory that the pattern of the file. and call the same pattern in to tFilelist.

1) pattern would be like this globalMap.put("pattern", "Customersdata_"+ TalendDate.getDate("yyyyMMdd") + pattern_suffixe);

2) use the same paatern to get see the below screen shot.

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

[TRF]

Hi,

Stricly based on your filenames, here is a complete solution to get the latest file:

tFileList is configured with the filemask "*.psv" and here is the content of the tJava component:

// get the date contained into the current filename
int dateFile = Integer.parseInt(((String)globalMap.get("tFileList_2_CURRENT_FILE")).replaceAll("^.*_", "").replace(".psv", ""));

// if there is no "maxDate" known or the current date is > to "maxDate" then the current date becomes the "maxDate"
if ((Integer)globalMap.get("maxDate") == null || dateFile > (Integer)globalMap.get("maxDate")) {
   globalMap.put("maxDate", dateFile);
   globalMap.put("lastFile", (String)globalMap.get("tFileList_2_CURRENT_FILE"));
}

Now you can get the filename using the global variable called "lastFile" using the following syntax (for example to populate the Filename field of a tFileInputDelimited or tFileInputFullRow component):

(String)globalMap.get("lastFile")

This example is based on your input and works fine.

You may have to adapt it to your case.

Don't forget to mark your case as solved as soon as it is ok for you (Kudo also accepted).

[Anonymous]

Hi,

I'm very thankful for your detailed explanation.

My filename is coming like this

Good_Customersdata_20171231_20171231092635.psv

In this file 20171231092635 is date with timestamp I can to check based on this.

Please help me to sort it 

[TRF · Accepted solution]

Just have to change the tJava code to use BigInteger instea of Int:

BigInteger dateFile = new BigInteger(((String)globalMap.get("tFileList_2_CURRENT_FILE")).replaceAll("^.*_", "").replace(".psv", ""));

if ((BigInteger)globalMap.get("maxDate") == null || dateFile.compareTo((BigInteger)globalMap.get("maxDate")) == 1) {
	globalMap.put("maxDate", dateFile);
	globalMap.put("lastFile", (String)globalMap.get("tFileList_2_CURRENT_FILE"));
}

[Anonymous]

It is showing BigInt cannot be resolved.