I read many articles on same concern I am opening this discussion but couldn't find the solution:
I have folder which has sub folders lets say 100 in count, i.e. SubFolder1, SubFolder2 SubFolder3 ...., each sub folder has few files, csv in format but same structure. Few csv files could have 0 records but headers are present.
So I need to load SubFolder1 files make a qvd and give name SubFolder1.qvd to it same for other sub folders lets say SubFolder2.qvd, SubFolder3.qvd....
I read the code and created qvds as subfolder1.qvd which has total records present in all files under subfolder1.
sub DoDir (Root) for each Ext in 'csv'
for each File in filelist (Root&'\*.' &Ext) Temp: LOAD '$(File)' as Name, // FileSize( '$(File)' ) as Size, // FileTime( '$(File)' ) as FileTime // autogenerate 1; * from $(File);
for each Dir in dirlist (Root&'\*' )
call DoDir (Dir)
let vDirectoryName = subfield('$(Dir)','/',5); NoConcatenate $(vDirectoryName): load * Resident Temp;
store $(vDirectoryName) into "lib://Data/MainFolder/$(vDirectoryName).qvd"(qvd); drop table $(vDirectoryName);
drop table Temp;
next Dir end sub
call DoDir ('lib://Data/MainFolder/');
Now, lets say in subfolder1 there is 10 files whose names are timestamp lets say
As per my current code, it loads all 10 files from subfolder1 and make subfolder1.qvd but now I am looking for incremental load where new file lets say 20190911 will come in subfolder1 and by incremental code, new Qvd which will have only data for 20190911 will append on the subfolder1.qvd which was generated by above code.