Loop Incremental Load with 2 different fields

cbaqir — Thu, 19 Oct 2017 20:31:16 GMT

I have a QVD that combines information from two types of files - one for secure messages and one for insecure. I want to be able to simply dump new files in the source folder and run the script to reload the data. I believe that I will have to then move the file out of the folder. As a bonus, it would be great if I didn't have to do that but that complicates this.

I want to use a loop in my script to add new source data files to my QVD. The problem is that I am loading 2 files with different fields.

I think that the solution is to possibly have two iterations of the loop script but using two different source file folders but I can't figure out the right way to do that. I assume that instead of using Load * I need to specify the field changes similar to the way I created the initial QVD file (see commented lines in script). Help!

Here is the loop script:

//Loop to load new secure files

Set vConcatenate = ;

sub ScanFolder(Root)

for each FileExtension in 'csv'

for each FoundFile in filelist( Root & '\*.' & FileExtension)

FileList:

$(vConcatenate)

LOAD *
, '$(FoundFile)' as SourceFile

FROM [$(FoundFile)] (txt, codepage is 1252, embedded labels, delimiter is ',', msq, header is 0 lines);

Set vConcatenate = Concatenate;

next FoundFile

next FileExtension

for each SubDirectory in dirlist( Root & '\*' )

call ScanFolder(SubDirectory)

next SubDirectory
end sub

Call ScanFolder('..\External_Data\Messages_Source_Files') ;

store Avizia into [..\QVD\01_Extract_QVD\Avizia.qvd]
(qvd);

topic Loop Incremental Load with 2 different fields in QlikView

Loop Incremental Load with 2 different fields