Issue in incremental load

[iswarya]

I store last 2 years of data in qvd and concatenate with daily refresh of 3 months data. But, after incremental load the values are not maching with the full load data. Any reason why the values not matching

[Mauritz_SA]

Hi @iswarya 

It is very difficult to say why. Incremental loading is a critical component of various enterprise solutions and works well - it all just depends on the implementation. Why do you need to load the past three months' data and not only the new or updated records? Also, are you accounting for deletions if you have?

This article explains the basic concepts pretty well.

I am sure if you post your logic then someone on the Community will be able to help.

Regards,

Mauritz

[HirisH_V7]

Could you please give much more info on which fields your'e considering for Incremental load i.e any key unique fields or date or any other.

Also let us know how many steps of increments your'e following : Insert, Update ,delete.

Better post some sample data which resembles your'e scenario, so that it will be easy for breaking it up.

[jonathandienst]

>>But, after incremental load the values are not maching with the full load data

The most likely problems are duplicates or gaps . If you are loading 3 months daily, how are you eliminating duplicates? That is what I would look at first.

[iswarya]

The data wont be updated or deleted. So I just concatenate the new data. 

[jonathandienst]

If you are loading 30 days every day, then are you not creating duplicates of the last 29 days data?

[Vegar]

I'm in line with @jonathandienst , you're probably experiencing some overlap (or gaps).

I reccomend you to take look at this Qlik video as a start. Basic Incremental Load - Qlik applications