Incremental load must not overwrite previous records

[Femke]

Hi all, 

I am stuck on this issue: I have a database for which I calculate the number of records (count(rowno()). Every day, the latest records have to be inserted, but not update the previous records. Just add the new data on that day. 

For example:

03/02 5000 records

04/02 5050 records

03/02 has to be kept. 

The problem I have is that '03/02' is overwritten by the new date. 

This date has to be left on the first row. 

I first loaded the entire dataset, then I picked the max number of records in that dataset, then I'm loading where the latest records differ from the max. 

Can anyone please help me? 

In attachment you find the incremental load. The first load is a load of the entire db. 

Thanks!!

Thanks!!

[cengizeralp]

Hi Femke_UZB,

We can comment better if you can post the incremental loading script part.

regards,

[prabhu0505]

Take a look at this, it might help...

https://help.qlik.com/en-US/sense/November2019/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-fil...

[johngouws]

Good morning. 

In your data source, is there a record as a timestamp or flagged as the 'active' record? 

Regards

John

[anushree1]

Do not use Not Exists Condition post Concatenate, in case you are using the Standard Incremental Procedure described on help site viz., https://help.qlik.com/en-US/sense/November2019/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-fil...

[jonathandienst]

Are you already doing this?

First load from the source with where date = today() or  where date >= today()

Then concatenate the qvd with where date  < today()

[rathore01]

Hi There,

Check this out and I am sure you will get your answers.

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

Cheers!

[Femke · Accepted solution]

Hi all, 

Thank you for your help! 

I found the solution. The first entire load has to be set in comment before loading the incremental data. At the end a concatenate function is required. 

Thanks again

Femke