Skip to main content
Announcements
SYSTEM MAINTENANCE: Thurs., Sept. 19, 1 AM ET, Platform will be unavailable for approx. 60 minutes.
cancel
Showing results for 
Search instead for 
Did you mean: 
NNot_defined1674577304
Contributor II
Contributor II

Talend job copy to s3

I have created a job that would copy files from directory to s3

bucket. It’s copying all the files to s3 but I need a start and end date selection in Tac and studio to pass context parameters to desired dates and tjava code should iterate through dates inside source file name like abcdefghi_20210401_123455.xml.

_20210401_ is the date I’m trying get range from source file list. So that I can select a date range suppose 2022-01-01 to 2022-12-01. Please suggest with contexts I need .

Labels (6)
2 Replies
anselmopeixoto
Partner - Creator III
Partner - Creator III

Hi @Not defined Not defined​ 

 

I guess you're already using a tFileList to iterate over the list of source files and put their path on tS3Put. If that's the case, I suggest you put a tJava in between those components so you can extract the date String from each file name and compare it with the context variables. Then your job will become something like this:

 

tFileList -- [iterate] --> tJava --- [If Trigger] --> tS3Put

 

So in tJava you will have something like this:

 

//converts the String extracted from file name to Date and puts in a global variable

globalMap.put("FileDate", TalendDate.parseDate("yyyyMMdd", ((String)globalMap.get("tFileList_1_CURRENT_FILE")).split("_")[1]);

 

And on the If trigger connecting tJava and tS3Put you will compare the date stored in the variable above with those in context:

 

TalendDate.compareDate(context.StartDate, ((Date)globalMap.get("FileDate))) == -1 && TalendDate.compareDate(context.StartDate, ((Date)globalMap.get("FileDate))) == 1

 

The TalendDate.compareDate() method "return the result whether two date is the same, if first one less than second one return number -1, equal as

 return number 0, bigger than return number 1.". There's also another implementation that takes a third parameter, allowing to compare only a date part like "dd" (day) for example.

 

 

 

NNot_defined1674577304
Contributor II
Contributor II
Author

I appreciate your quick response. This code is not copying source files to s3put. Can please suggest code modifications and contexts needed to support the process? Thanks