topic Re: Incremental load must not overwrite previous records in App Development

Incremental load must not overwrite previous records

Femke — Tue, 04 Feb 2020 10:45:13 GMT

Hi all,

I am stuck on this issue: I have a database for which I calculate the number of records (count(rowno()). Every day, the latest records have to be inserted, but not update the previous records. Just add the new data on that day.

For example:

03/02 5000 records

04/02 5050 records

03/02 has to be kept.

The problem I have is that '03/02' is overwritten by the new date.

This date has to be left on the first row.

I first loaded the entire dataset, then I picked the max number of records in that dataset, then I'm loading where the latest records differ from the max.

Can anyone please help me?

In attachment you find the incremental load. The first load is a load of the entire db.

Thanks!!

Re: Incremental load must not overwrite previous records

cengizeralp — Mon, 03 Feb 2020 19:10:27 GMT

Hi Femke_UZB,

We can comment better if you can post the incremental loading script part.

regards,

Re: Incremental load must not overwrite previous records

prabhu0505 — Mon, 03 Feb 2020 22:23:20 GMT

Take a look at this, it might help...

https://help.qlik.com/en-US/sense/November2019/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-files-incremental-load.htm

Re: Incremental load must not overwrite previous records

johngouws — Tue, 04 Feb 2020 03:07:42 GMT

Good morning.

In your data source, is there a record as a timestamp or flagged as the 'active' record?

Regards

John

Re: Incremental load must not overwrite previous records

anushree1 — Tue, 04 Feb 2020 05:36:09 GMT

Do not use Not Exists Condition post Concatenate, in case you are using the Standard Incremental Procedure described on help site viz., https://help.qlik.com/en-US/sense/November2019/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-files-incremental-load.htm?_ga=2.245950468.1849336005.1580715277-1828563603.1575179477

Re: Incremental load must not overwrite previous records

jonathandienst — Tue, 04 Feb 2020 06:17:41 GMT

Are you already doing this?

First load from the source with where date = today() or where date >= today()

Then concatenate the qvd with where date < today()

Re: Incremental load must not overwrite previous records

rathore01 — Tue, 04 Feb 2020 06:37:35 GMT

Hi There,

Check this out and I am sure you will get your answers.

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

Cheers!

Re: Incremental load must not overwrite previous records

Femke — Fri, 07 Feb 2020 10:04:00 GMT

Hi all,

Thank you for your help!

I found the solution. The first entire load has to be set in comment before loading the incremental data. At the end a concatenate function is required.

Thanks again

Femke