Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

Announcements
BI & Data Trends 2021. Discover the top 10 trends emerging in today. Join us on Dec. 8th REGISTER
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Specialist III
Specialist III

Load csv files from different folders

Dear techies,

I read many articles on same concern I am opening this discussion but couldn't find the solution:

I have folder which has sub folders lets say 100 in count, i.e. SubFolder1, SubFolder2 SubFolder3 ...., each sub folder has few files, csv in format but same structure. Few csv files could have 0 records but headers are present.

So I need to load SubFolder1 files make a qvd and give name SubFolder1.qvd to it same for other sub folders lets say SubFolder2.qvd, SubFolder3.qvd....

Can anyone please help me how to achieve it

 

Thanks,

Rohit

 

Labels (1)
2 Replies
Highlighted
Creator II
Creator II

Highlighted
Specialist III
Specialist III

Thanks for the instant reply.

I read the code and created qvds as subfolder1.qvd which has total records present in all files under subfolder1. 

sub DoDir (Root)
for each Ext in 'csv'

for each File in filelist (Root&'\*.' &Ext)
Temp:
LOAD
'$(File)' as Name,
// FileSize( '$(File)' ) as Size,
// FileTime( '$(File)' ) as FileTime
// autogenerate 1;
*
from $(File);

next File

next Ext


for each Dir in dirlist (Root&'\*' )

call DoDir (Dir)

let vDirectoryName = subfield('$(Dir)','/',5);
NoConcatenate
$(vDirectoryName):
load
*
Resident Temp;

store $(vDirectoryName) into "lib://Data/MainFolder/$(vDirectoryName).qvd"(qvd);
drop table $(vDirectoryName);

drop table Temp;

next Dir
end sub

call DoDir ('lib://Data/MainFolder/');

 

Now, lets say in subfolder1 there is 10 files whose names are timestamp lets say

20190901,

20190902,

...

20190910

As per my current code, it loads all 10 files from subfolder1 and make subfolder1.qvd but now I am looking for incremental load where new file lets say 20190911 will come in subfolder1 and by incremental code, new Qvd which will have only data for 20190911  will append on the subfolder1.qvd which was generated by above code.

Can you please add your input ?

 

Thanks,

Rohit