topic Re: WHat is incremental load ?? in QlikView

WHat is incremental load ??

nikhilgarg — Thu, 04 Sep 2014 06:26:52 GMT

HEy,

WHat is incremental load ?? WHy it is used ?? Can any one give an example ??

Re: WHat is incremental load ??

its_anandrjs — Thu, 04 Sep 2014 06:30:54 GMT

You can search Qlik community for this and get many post related to this.

Re: WHat is incremental load ??

tresB — Thu, 04 Sep 2014 06:36:47 GMT

One of them:

Incremental Load

Re: WHat is incremental load ??

its_anandrjs — Thu, 04 Sep 2014 06:43:07 GMT

You can read this useful document also for get an idea for Incremental load.

Re: WHat is incremental load ??

MayilVahanan — Thu, 04 Sep 2014 06:51:42 GMT

Please refer this

http://community.qlik.com/search.jspa?q=incremental+load#type=document

Additional with above suggestion:

Incremental Load Scenarios.pdf

Re: WHat is incremental load ??

nikhilgarg — Thu, 04 Sep 2014 07:38:37 GMT

Hey ,

I read the ppt at this given link but i still didnt get the perfect idea why incremental load is used ?? Is it used for doing operations on log files only ??

Please help me understanding incremental load.

Thanx.

Re: WHat is incremental load ??

nikhilgarg — Thu, 04 Sep 2014 07:38:54 GMT

Hey ,

I read the ppt at this given link but i still didnt get the perfect idea why incremental load is used ?? Is it used for doing operations on log files only ??

Please help me understanding incremental load.

Thanx.

Re: WHat is incremental load ??

Thu, 04 Sep 2014 07:40:33 GMT

Incremental load --> appending delta on history data instead of fetching all data from source system

Re: WHat is incremental load ??

Siva_Sankar — Thu, 04 Sep 2014 08:36:12 GMT

Nikhil,

Consider the following example..

A company may have two platforms, one that processes orders, and a seperate accounting system. The accounts department enters new customer details into the accounting system but has to ensure these customers appear in the order processing system.

To do this it runs a nightly batch job that sends data from the accounting system to the order system.

If they were deleting all customer details in the order system and refilling with all the customers in the accounting system then they would be performing a full reload which takes long time.

If they only sent accross the new customers and the customers that had been changed they would be performing an incremental load which means the incremental load will keep the old data as it is and load only new data. Consumes less load time.

Find the attachment for detail information

Re: WHat is incremental load ??

nikhilgarg — Thu, 04 Sep 2014 08:55:08 GMT

Hey,

But to reduce load time and load new data , partial load can be used. Right ??

Re: WHat is incremental load ??

ashfaq_haseeb — Thu, 04 Sep 2014 09:02:41 GMT

Hi,

start from here.

Regards

ASHFAQ

Re: WHat is incremental load ??

Siva_Sankar — Thu, 04 Sep 2014 09:06:26 GMT

Nikhil,

Partial reload executes a specific part of sciprt, this can for example be used to refresh a certain table.

Incredemental reload is defined in the script to update certain tables incredementally, for example to add only new transactions to a transaction table.

Incremental reload is especially used to decrease the reload time, by for instance only loading new records from a data source instead of the complete table.

Re: WHat is incremental load ??

Siva_Sankar — Thu, 04 Sep 2014 09:07:53 GMT

Check here Partial load vs incremental reload ??

Re: Re: WHat is incremental load ??

tresB — Thu, 04 Sep 2014 09:09:29 GMT

There are some substantial differences between incremental load and partial load. Goto page no. 27 of the attached doc.

LOAD data into QlikView

Re: WHat is incremental load ??

nikhilgarg — Thu, 04 Sep 2014 09:44:45 GMT

Hey anand,

Does incremental load applies with these cases only ??

Insert , insert and update , insert , update and delete ??

Re: WHat is incremental load ??

Siva_Sankar — Thu, 04 Sep 2014 09:55:34 GMT

Below are the incremental reload scenarios:

1) Case 1: Append Only (typically log files

2) Case 2: Insert Only (No Update or Delete)

3) Case 3: Insert and Update (No Delete)

4) Case 4: Insert, Update and Delete

Re: Re: WHat is incremental load ??

Sat, 06 Sep 2014 16:59:32 GMT

Hello Nikhil Garg,

Incremental load is a common task in relation to data bases. purpose of incremental load is, To Extract only the new or modified transactions from the source table and append those records to the ones previously saved in the Base QVD's. The approach of incremental load depends upon the nature of the source database, meaning

IF you want to use the data from a logfile which is contained in a textfile, you can use the append only,no need to connect to ODBC OR OLE DB, QV keeps track of the number of records that have been previously read and loads only added at the end of the files.

If the data resides in a database other than a logfile you can use insert, update and delete(I,U &D)

The conditions apply for I,U and D

a. The data source can be any database

b.QV loads records inserted into the database or update in the database after the last script execution.

c. QV removes records deleted from the database after the last script execution.

d.A field ModificationDate is required to recognize which records are new.

e. Primary key field is required to sort out updated records from the QVDfiles.

The reading of QVD file will be forced to standard mode rathan than super-fast mode but still it is faster than loading the entire database.

Thanks

Sweety

Re: WHat is incremental load ??

Fri, 29 May 2015 11:19:00 GMT

In simple words,Incremental lod is delta load.

Re: WHat is incremental load ??

Fri, 29 May 2015 12:13:35 GMT

Hi,

In many business environments, there will be many large transaction tables. These tables may receive 1000’s or millions of rows of data daily or monthly. Since the data is huge and it may be sensitive also from the history standpoint, you may not want to delete the existing records, instead you will load only if the data is new. In such scenarios incremental load is used.

Thanks.