Comparing filelist to table on load

[#1]

Hi

I am (incrementally) loading data from CSV-files, located in different folders.

Each day, more (hundreds of) csv-files will be added to these folders, and they will not (cannot) be (re)moved after load.

To loop through the folders I am using the "Sub DoDir(Root)" from the user's guide

And the using:

// CODE

For Each File In FileList  ('$(FileRoot)')

          LOAD...

          variable_1,

          variable_2,

          etc...

          Form [$(File)](txt)

     <lots of code and reloads follows>

Next File

// CODE END

to load data from the csv-files therein.

NOW - I also have a qvd-file with a table with the names of the files previously processed.

Fileldname is Previous_filenames

What I need to do is to compare the filename of each csv-file to the list of previously loaded files, and only load the ones not previously loaded.

Something like:

If not exists(Previous_filenames, Filename('$(File)') Then...

Any ideas?

[whiteline · Accepted solution]

You've already done half of it, assigned a name.

Now create a field Previous_filenames, just add it to your LOADs:

LOAD...

          variable_1,

          variable_2,

  SubField('$(File)', '\', Substringcount('$(File)')+1) as Previous_filenames

Form [$(File)](txt)