topic How to skip files? in QlikView

How to skip files?

Anonymous — Tue, 12 Sep 2017 06:25:44 GMT

Hello,

I have created somewhat of a logic to my QlikView application, where I have stored the filenames on a text file that has been already read.

And then on the editor script, I just check if that has already been on the list. But the load always still tries to "read" the file and shows the file name on the load screen and shows just that it does not load any rows from it. So actually it does not skip it on the way I would like, this current way takes too much time on my mind. I would not like to have it even opening the file at all, just skip it right away.

script:

---------

// Find files that has already read

distinctFileNames:

load distinct filenameOfFile as fof from \\blaablaa\blaa.qvd(qvd);

// Load files that are new and not already read and stored to QVD

for each file in filelist('\\blaablaa\blaablaablaa_*.csv')

if filesize(file) > 0 then

table1:

load *,

FileName() as filenameOfFile,

from

$(file)

(txt, utf8, embedded labels, delimiter is ',', msq)

where not EXISTS(fof,FileName()) ;

endif

NEXT;

---------

Any ideas that would be better, than the WHERE NOT EXISTS part so that the load would not even check the file..?

Cheers,

Niko

Re: How to skip files?

prieper — Tue, 12 Sep 2017 07:57:36 GMT

You do not need to open the files, just check the results from FILELIST.

aircode might be:

ExistingFiles: LOAD FileName FROM Files.QVD; // List of already digested files

FOR EACH sFile IN FILELIST('....')

NewFiles:

LOAD

'$(sFile)' AS NewFileName

AUTOGENERATE 1

WHERE

NOT EXISTS(FileName, '$(sFile)'; // load only the not-existing files

NEXT sFile

NewData:

FOR i = 0 TO NOOFROWS('NewFiles') // go through the list of new files, one-by-one

LET sNewFile = PEEK ('NewFileName', 0, 'NewFiles')

LOAD * FROM $(sNewFile);

CONCATENATE (ExistingFiles) LOAD '$(sNewFile)' AS FileName AUTOGENERATE 1; // add to existing list

NEXT i

STORE ExistingFiles INTO Files.QVD; // overwrite

Re: How to skip files?

Anonymous — Tue, 19 Sep 2017 10:44:21 GMT

Hello Peter!

And thank you for your help, this idea did help me to get my solution to work.

Had to do some external job but when I did get all in place, it did do the job as I wanted.

Thank you for the idea-sharing help, much appreciated!

Cheers,

Niko