Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

zahidrahim_ocp
Contributor

Incremental Load on QVD

Dear Experts,

I have generated few QVD against few source tables and make dashboard for a given date. Now i want to setup incremental loads on these QVDs in Qlik Sense.

Please guide and help with step by step instructions as i am doing it for the first time.

Regards,

Zahid Rahim

23 Replies
micheledenardi
Valued Contributor

Re: Incremental Load on QVD

Hi Zahid,

there are lots of articles in the web that explain the approach to Incremental loads:

Incremental load for beginner with example

Incremental Load

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

QlikView Incremental Load

https://www.quickintelligence.co.uk/qlikview-incremental-load/

First of all look at these articles, if you have some more specific questions the community is always ready to help you.

omarbensalem
Esteemed Contributor

Re: Incremental Load on QVD

Suppose you have this table to begin with :

table:

Load * from source where date <today();

//Now we loaded everything but to day; and after that, we want to only load today()'s data and concatenate it with the rest

First thing to do is creating the first QVD:

Store table into path.qvd(qvd);

With that being done, comment your script and change it as follow:

table:

Load * from source where date=today();

concatenate

load * from path.qvd(qvd);

store table into path.qvd(qvd);

With that, we load today's data, than concatenate it with the <tody()'s data (previously stored in the qvd)

We'll thus have a new table containing today's and previous days data. we store it into the same path.

With that when we reload, we'll always have the same process.

Hope this helps,

Omar

zahidrahim_ocp
Contributor

Re: Incremental Load on QVD

Dear Omar,

Thank you for the reply.

By looking at the example.

It means we must setup it 12:00AM for complete day transactions what about if we perform it on a specific time of the day?

Secondly it loads only the new data what about the changed data in the past?

Best Regards,

Zahid Rahim

tajmohamed30
Contributor III

Re: Incremental Load on QVD

to reload specific time of the day you can go for autosys control m links below

Reload with Control-M

Qlikview batch job integration with autosys job... | Qlik Community

zahidrahim_ocp
Contributor

Re: Incremental Load on QVD

Is there any way to update changes happened with the past data?

tajmohamed30
Contributor III

Re: Incremental Load on QVD

in DB you can find the max of last modified date, in qvw you can use =reloadtime() in text object

zahidrahim_ocp
Contributor

Re: Incremental Load on QVD

Dear Taj,

Thank you for your reply. I was looking at the below script and not understanding how can we determine the last executed load time.

QV_Table:

SQL SELECT PrimaryKey, X, Y FROM DB_TABLE

WHERE ModificationTime >= #$(LastExecTime)#;

Can you please guide me with some code.

Regards,

Zahid Rahim

tajmohamed30
Contributor III

Re: Incremental Load on QVD

hope the below link will be useful

Incremental Load Script

zahidrahim_ocp
Contributor

Re: Incremental Load on QVD

Dear Taj,

Thank you for the link it was really useful. But still confuse how to get and use #$(LastExecTime)# As this thing #$(LastExecTime)# is really very much new to me.


Regards,


Zahid Rahim