Reading the latest data:

[odaher123]

Hi guys,

Every day an application saves new data in new files (csv).

How to read only the data from the last files deposited since the last reading?

I managed to create a table named Listing_of_files.

Thanks for your help

/////////////////////////////////////////////////////////////////////////

set file_path_csv = 'F:\Statistiques\csv';

sub DoDir (Root)

    For Each Ext in 'qvd', 'csv'

        For Each File in filelist (Root&'\*.' &Ext)

        Listing_of_files:

        LOAD

        '$(File)' as Name, FileSize( '$(File)' ) as

        Size, FileTime( '$(File)' ) as FileTime

        autogenerate 1;

        Next File

    Next Ext

    For Each Dir in dirlist (Root&'\*' )

        Call DoDir (Dir)

    Next Dir

End Sub

Call DoDir ('$(file_path_csv)');

///////////////////////////////////////////////////

how to do next?

[dplr-rn]

Step 1 - Set up the csv file load (i.e. load only the new files)

Basically for the first load (based on a flag) load the file names and data into a qvds

Subsequent loads

- get list of files names

- load old list of file names from qvd

- create another table with latest filenames only

- load data from qvd

- loop through new file names and load which gets autoconcatenated

- drop unnecessary tables (havent done this in the app to prove it works)

- save qvds

see attached script

Step 2 - implement incremental load (update and/or) delete on the latest loads. check below

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/