topic Re: INCREMENTAL LOAD in QlikView

INCREMENTAL LOAD

d_koti24 — Wed, 11 Jun 2014 12:33:55 GMT

I WANT KNOW ABOUT THE INCREMENTAL LOAD

Re: INCREMENTAL LOAD

arulsettu — Wed, 11 Jun 2014 12:35:38 GMT

check this

Incremental Load in QlikView – Part1 | Learn QlikView

Re: INCREMENTAL LOAD

Gysbert_Wassenaar — Wed, 11 Jun 2014 12:36:46 GMT

See this discussion: Dynamic Update and Incremental Loading

Re: INCREMENTAL LOAD

MayilVahanan — Wed, 11 Jun 2014 12:37:13 GMT

Please refer this link

http://community.qlik.com/docs/DOC-4597

Incremental Load.docx

Re: INCREMENTAL LOAD

its_anandrjs — Wed, 11 Jun 2014 12:57:23 GMT

You can read this best document also for the Incremental load

Re: INCREMENTAL LOAD

ramasaisaksoft — Wed, 11 Jun 2014 13:02:13 GMT

Hi Kote,

incremental load is nothing but you load the data into u r .QVW(Qlikview Window) file by .QVD files(Qlik view Document ) just like u load .CSV files and Database files(Tables)

How to create a test.QVD file is

load * from emp_table;

select * from emp_table;

Store emp_table into D:\QVD_files \test.qvd;//it is a command for creating the QVD file.

So QVD file is stored in D drive QVD_files folder and the name as test.QVD but u can't open these files directly ,these files u can use in QVW files.

How to Use tset.QVD file in incremental. QVW file

load * from D:\QVD_files \test.qvd (qvd);

this entire process is called Incremental Load.

if u need any other details please contact me on ramasaiqvw@gmail.com at any time.(i will speak Telugu also).

Re: INCREMENTAL LOAD

d_koti24 — Wed, 11 Jun 2014 13:04:15 GMT

K....thank u boss

Re: INCREMENTAL LOAD

Wed, 11 Jun 2014 13:05:00 GMT

Hey Please help ….. I am getting all the emails on my working email address ..do you know where can I de-Activate this ?

Re: INCREMENTAL LOAD

d_koti24 — Wed, 11 Jun 2014 13:17:19 GMT

i dont knw boss

Re: INCREMENTAL LOAD

its_anandrjs — Wed, 11 Jun 2014 13:17:38 GMT

Hi,

Let me clear about the Incremental load There are 3 types of the Incremental load

Insert

Update

Delete

But depends on the scenarios how you use it

Step One:- Load Initial table

TableName:

Load * from Location;

Store TableName into TableName.qvd(qvd);

Step Two:- Insert based on the Primary keys

MaxIDTable:

LOAD Max([Indentity Number]) as MaxIncrID

FROM CUSTOMERS.xls

LET vMaxIncrID = peek('MaxIncrID',0,'MaxIDTable');

Step Three:-

TableName:

LOAD *

From Raw Source Location

WHERE [Indentity Number] >= $(vMaxIncrID);

Concatenate

LOAD

FROM

TableName.qvd (qvd);

STORE TableName into TableName.qvd(qvd);

DROP Table MaxIDTable;

Re: INCREMENTAL LOAD

d_koti24 — Wed, 11 Jun 2014 13:18:56 GMT

thank u boss............

Re: INCREMENTAL LOAD

Anonymous — Thu, 28 Aug 2014 09:45:22 GMT

Rather than using incremental loads you could write a server side script to pull your data from the DB and save as a CSV file, incremental loads can become slow the bigger the table.

A summary for you

Server side script pulls data from DB and saves as CSV files.
QVW file loops over CSVs and creates a mirror QVD file.
All of you reports can pull in the QVDs - LOAD * FROM myUsersCsv_*.qvd (qvd);

An example of speed savings:

To rebuild a 35M row x 40 Column User db table into QV via incremental loads - total build 3.5 hours
To rebuild same table into CSV 8 mins, then point 2 of summary 10 mins. - total build 18 minutes

I've written about it in more detail:

http://community.qlik.com/message/596536

Remeber "Incremental loads are dead, long live the CSV"