Skip to main content
Announcements
See what Drew Clarke has to say about the Qlik Talend Cloud launch! READ THE BLOG
cancel
Showing results for 
Search instead for 
Did you mean: 
sanket94
Contributor III
Contributor III

Load only the latest file from the directory

Scenario:

I receive 2 types of file a Po file and Ship file daily twice. the file names are abbreviated as 

File1: _DailyStatusPO[Date]_[time24hrs]

File2: _DailyStatusSHIP[Date]_[time24hrs]

I  want to load only the file that is latest; I do not receive these files on Holidays and weekends so load the latest in those cases. Could someone please help with this.

11ca.JPG

 

 

Labels (3)
1 Reply
martinpohl
Partner - Master
Partner - Master

here's an example, maybe some bugs in but as the steps to go:

for each vType in ('PO','SHIP');   // add more if available

for each vFile in filelist ([LIB//NAME/dailystatus$(vType)*.dat]  // do a loop for all files with filetype

// only load filename and date from filename

Files:

first 1 load

filebasename() as File,

date(date#(right(substring(filebasename(),'_',2),8),'MMDDYYYY'),'YYYYMMDD') as Fileday

from '$(vFile)';

next

//sort datas by date desc

Filesorder:

noconvatenate load

File,

Fileday

resident Files

order by Fileday DESC;

// write the newest (the first) value in a variable

let vLoadfile = peek('File'),0,'Filesorder');

// and then to a table with all table to load from

Loadfiles:

load

'$(vLoadfile)' as Loadfile

autogenerate (1);

drop table Filesorder;

drop table Files;

next

// when ready do a loop for all files to load

let vCountfiles = noofrows(Loadfiles)-1;

for vload = 0 to $(vCountfiles)

let vFinalfile = peek('Loadfile',$(vload),'Loadfiles');

load 

from '$(vFinalfile)';

next

drop table Finalfile;