Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
How to get the latest file. when we run history. Can anyone please help to sort it out.
Just have to change the tJava code to use BigInteger instea of Int:
BigInteger dateFile = new BigInteger(((String)globalMap.get("tFileList_2_CURRENT_FILE")).replaceAll("^.*_", "").replace(".psv", "")); if ((BigInteger)globalMap.get("maxDate") == null || dateFile.compareTo((BigInteger)globalMap.get("maxDate")) == 1) { globalMap.put("maxDate", dateFile); globalMap.put("lastFile", (String)globalMap.get("tFileList_2_CURRENT_FILE")); }
There is only modified in tfilelist.
In my scenario I'm downloading all the history files in job server, but based on filedate it should process only latest date file.
Eg: 1.Customersdata_20180126.psv
2.Customersdata_20180201.psv
3.Customersdata_20171231.psv
We have to process only 2.Customersdata_20180201.psv file which is latest
Thanks,
Venkat
Can you please explain in detail, because file names may vary.
first put in the memory that the pattern of the file. and call the same pattern in to tFilelist.
1) pattern would be like this globalMap.put("pattern", "Customersdata_"+ TalendDate.getDate("yyyyMMdd") + pattern_suffixe);
2) use the same paatern to get see the below screen shot.
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Hi,
Stricly based on your filenames, here is a complete solution to get the latest file:
tFileList is configured with the filemask "*.psv" and here is the content of the tJava component:
// get the date contained into the current filename
int dateFile = Integer.parseInt(((String)globalMap.get("tFileList_2_CURRENT_FILE")).replaceAll("^.*_", "").replace(".psv", ""));
// if there is no "maxDate" known or the current date is > to "maxDate" then the current date becomes the "maxDate"
if ((Integer)globalMap.get("maxDate") == null || dateFile > (Integer)globalMap.get("maxDate")) {
globalMap.put("maxDate", dateFile);
globalMap.put("lastFile", (String)globalMap.get("tFileList_2_CURRENT_FILE"));
}
Now you can get the filename using the global variable called "lastFile" using the following syntax (for example to populate the Filename field of a tFileInputDelimited or tFileInputFullRow component):
(String)globalMap.get("lastFile")
This example is based on your input and works fine.
You may have to adapt it to your case.
Don't forget to mark your case as solved as soon as it is ok for you (Kudo also accepted).
Hi,
I'm very thankful for your detailed explanation.
My filename is coming like this
Good_Customersdata_20171231_20171231092635.psv
In this file 20171231092635 is date with timestamp I can to check based on this.
Please help me to sort it
Just have to change the tJava code to use BigInteger instea of Int:
BigInteger dateFile = new BigInteger(((String)globalMap.get("tFileList_2_CURRENT_FILE")).replaceAll("^.*_", "").replace(".psv", "")); if ((BigInteger)globalMap.get("maxDate") == null || dateFile.compareTo((BigInteger)globalMap.get("maxDate")) == 1) { globalMap.put("maxDate", dateFile); globalMap.put("lastFile", (String)globalMap.get("tFileList_2_CURRENT_FILE")); }
It is showing BigInt cannot be resolved.