0 Replies Latest reply: Oct 19, 2017 5:31 PM by Cassandra Baqir RSS

    Loop Incremental Load with 2 different fields

    Cassandra Baqir

      I have a QVD that combines information from two types of files - one for secure messages and one for insecure. I want to be able to simply dump new files in the source folder and run the script to reload the data. I believe that I will have to then move the file out of the folder. As a bonus, it would be great if I didn't have to do that but that complicates this.

       

      I want to use a loop in my script to add new source data files to my QVD. The problem is that I am loading 2 files with different fields.

      I think that the solution is to possibly have two iterations of the loop script but using two different source file folders but I can't figure out the right way to do that. I assume that instead of using Load * I need to specify the field changes similar to the way I created the initial QVD file (see commented lines in script). Help!

       

      Here is the loop script:

      //Loop to load new secure files

      Set vConcatenate = ;

      sub ScanFolder(Root)

      for each FileExtension in 'csv'

      for each FoundFile in filelist( Root & '\*.' & FileExtension)

      FileList:

      $(vConcatenate)

      LOAD * 
      , '$(FoundFile)' as SourceFile


      FROM
      [$(FoundFile)] (txt, codepage is 1252, embedded labels, delimiter is ',', msq, header is 0 lines);

      Set vConcatenate = Concatenate;

      next FoundFile

      next FileExtension

      for each SubDirectory in dirlist( Root & '\*' )

      call ScanFolder(SubDirectory)

      next SubDirectory
       
      end sub

       

       

       

      Call ScanFolder('..\External_Data\Messages_Source_Files') ;

      store Avizia into  [..\QVD\01_Extract_QVD\Avizia.qvd]
      (
      qvd);