Incremental Load based on multiple daily diifferent dates files

[101010]

Hi All,

I have a daily files for example SALES_20200201,SALES_20200202,SALES_20200203 and inside on this files consists of records from multiple different dates 

SALES_20200201 file

Date	No of records
31/01/2020	10000
30/01/2020	2000
29/01/2020	50
01/02/2020	10
02/02/2020	2

SALES_20200202 file

Date	No of records
31/01/2020	15000
30/01/2020	3000
29/01/2020	70
03/02/2020	20
04/02/2020	10

Wanted to split these files into multiple different qvd with respective date accordingly. For eg. NewSales_20200131.qvd consists of only records with date 31/01/2020.

After that wanted to perform incremental load based on each of newly created qvd by each of dates respectively for SALES_20200202 , SALES_20200203. If new inserted/updated records then append on existing newly created qvd else then store into new qvd with date.

1. Split and store from daily files into qvd with different dates.

2.Perform incremental load on the newly created qvd.

Appreciate if someone could help on this.

[IamAlbinAntony]

For insert and update you need the following,

A ModificationTime field (or similar) is required for Qlik Sense to recognize which records are new.

A primary key field is required for Qlik Sense to sort out updated records from the QVD file.

Example:  

QV_Table:

Load PrimaryKey, X, Y FROM SALES_TABLE

WHERE ModificationTime >= #$(LastExecTime)#;

Concatenate

LOAD PrimaryKey, X, Y FROM File.QVD

WHERE NOT Exists(PrimaryKey);

STORE QV_Table INTO File.QVD;