topic Load files within path in a table - ¿Any other better way? in App Development

Load files within path in a table - ¿Any other better way?

Fernando_Sanchez — Mon, 24 Oct 2022 10:50:03 GMT

Hi all,

I want to load in a table all files within a certain path.

For doing so, the script is easy.

FOR EACH vFile IN FileList ('path\')

let vFileName = filebasename ($(vFile));

FILES:
load * inline [
fileName
$(vFileName)];

next;

The problem I am facing is that sometimes there ara over 100.000 files within the path, and it takes too long.

Any other better way to implement it?

Regards

Re: Load files within path in a table - ¿Any other better way?

vinieme12 — Tue, 25 Oct 2022 09:30:46 GMT

you can load all files with the same file format in one go using wildcard ( * ) instead of name

You don't need FOR LOOP

If the table structure for all files in the same, QLIK will auto concatenate all tables

as below

All QVD files in a given folder path

FILES:
load *

From [lib://somepath/somefolder/*.qvd](qvd);

All XLSX files in path

FROM [lib://somepath/somefolder/*.xlsx]
(ooxml, embedded labels, table is sheetnamehere) ;

Re: Load files within path in a table - ¿Any other better way?

Fernando_Sanchez — Thu, 27 Oct 2022 21:10:18 GMT

Thanks a lot for the answer.

By doing so it loads all info within every file. I just need the name of the field. But you gave me an idea.

To achieve this you can, for example

FILES:
load first 1

filename() as fileName

From [path];

Unfortunatly, it takes too long as well. More or less the same with the loop.

The problem I am facing is that within a certain path i might have over 100.000 files, and ir order to get every filename it may take 1hour.

Any other quiker way?

Re: Load files within path in a table - ¿Any other better way?

Fernando_Sanchez — Thu, 27 Oct 2022 21:33:19 GMT

Interesting,

by simply not renaming fileName() it speeds up the load x 10...

FILES:
load first 1

filename()

From [path];

FILES_2:

load [fileName()] as file

Resident FILE;

drop table FILE_2

Re: Load files within path in a table - ¿Any other better way?

Fernando_Sanchez — Thu, 27 Oct 2022 21:41:16 GMT

Anyhow there is a problem when the file you are loading is not "loadable", for instance a PDF.

Anyone knows how to achieve this?

Regards.