topic Re: Rapid directory/file scan in QlikView

Rapid directory/file scan

Anonymous — Thu, 27 Jun 2013 05:24:18 GMT

Hi there,

I have to scan a directory with subfolders for each day of the year, containing up to around 400 files. So this will generate more than 10000 files each month. I tried a script using the code sequence "For each ... in ... ". That works very fine, but with very low speed (several hours). Is there a more rapid way e.g. using a macro and system functions to speed up the process. The goal is, to generate a list of files with the complete path, so that I can sort out the ones, for loading the complete data from the files.

Thanks a lot

Michael

Re: Rapid directory/file scan

Gysbert_Wassenaar — Thu, 27 Jun 2013 05:53:54 GMT

Something like this will get you a table with the full file paths:

execute cmd.exe /c dir /s /b d:\qvdata > myfilelist.txt;

LOAD @1 as Filename

FROM

myfilelist.txt

(txt, codepage is 1252, no labels, delimiter is ';');

You can use the subfield and./or index functions to get the directory and file names.

Re: Rapid directory/file scan

Anonymous — Thu, 27 Jun 2013 06:14:17 GMT

Hi Gysbert,

your idea makes the difference. It only took around 6 minutes to create a filellist with more than 900000 entries. Thanks a lot for that.

Michael