topic Re: incremental load in QlikView

incremental load

manoj217 — Wed, 25 Nov 2020 16:16:04 GMT

HI,

i have 3yrs (2011,2012,2013)of data in the application while doing the incremental load 1 more year (2014)data is added now the performance is reduced so any options to increase the performance. if i do the same function (adding 1yr (2015)more)again data will increase performance will reduce any suggestions.

Re: incremental load

Anil_Babu_Samineni — Thu, 07 Dec 2017 07:17:29 GMT

Don't reload full. Try to reload Partial Reload using ADD prefix for now.. Also, You can think Incremental Load concept to load only recent data with Club into old data

Re: incremental load

manoj217 — Thu, 07 Dec 2017 07:18:53 GMT

But data is increased in the Qlikview memory

Re: incremental load

stabben23 — Thu, 07 Dec 2017 07:19:21 GMT

Hi,

are you using the incremental load in a ETL loading process or in Direct in Your Dashboard Application?

Re: incremental load

manoj217 — Thu, 07 Dec 2017 07:26:11 GMT

in application . we have created the incremental load and actions performed in qmc on every month data is increasing and old data is also existed in the qlik memory so now the performance is reduced

Re: incremental load

Anonymous — Thu, 07 Dec 2017 07:26:55 GMT

incremental load is the option you might look for

Re: incremental load

Anil_Babu_Samineni — Thu, 07 Dec 2017 07:32:11 GMT

Either this is from QMC not the Qlikview. Recently, One of my colleague faced the issue.

My side, The issue is Every reload happens the File size (QVW) got increased. Does this same your question? And this happened for me in QMC. Will it so?

Re: incremental load

Anonymous — Thu, 07 Dec 2017 07:32:42 GMT

As you have mentioned you have the problem in dashboard, instead of reloading.

specially you have an issue with qlikView consuming more memory.

in that case, follow the below steps:

you cannot skip any year of data, so skip the unwanted columns in data model.
identify the dimensions and facts to create a schema, so that you can easily eliminate duplicates.
check for synthetic keys specially.
if you have tried these things, but you need to keep that big data model in your memory, all you can do is taking care of your expressions and limit the values in chart

Hope it helps

Hari.

Re: incremental load

manoj217 — Thu, 07 Dec 2017 07:33:11 GMT

if i want only recent 3 yr data and i want to delete the previous years data by doing the incremental load what will be code

Re: incremental load

Anonymous — Thu, 07 Dec 2017 07:38:25 GMT

We don;t know your field names, so we cannot write code for you,

but i suggest you to use where condition while loading the data, and use the recent three years as values

Re: incremental load

stabben23 — Thu, 07 Dec 2017 07:54:13 GMT

So you are not using QVD's in Your incremental load?