New Data

[qv_testing]

HI All,

Can we get New data without Incremental load.

like use last execution date..

I have million's of records.. How can we reload only new data?????

Like

Let vLastReload=  ReloadTime();

Load CategoryID,

        CategoryName,

        Price,

        Model

From Category;

Can we use variable in where Condition ??

Thanks in Advance...

[Gysbert_Wassenaar]

You can use a technique that's called incremental loading. See here, here and here for more information.

And to make things a lot easier to implement read Rob's blog post on incremental loading using qlikview components.

[Kushal_Chawda]

see this

http://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

[Chanty4u]

did u try with  partial reload?

by using

Add

[jagan]

Hi,

You have to load the new data with WHERE condition, but if you require old data in Dashboard then you have save the old data in QVD (for faster reloading) and then need to concatenate to the new data everytime.

Regards,

Jagan.

[qv_testing]

HI All,

Please anyone can help me.. I am Confusing..

First time implementing Incremental Load..

This is my table..

TestTable:

Load

    Category ID,

    Category,

    Product,

    DateCompleted,

    Price

from Category;

How to do Incremental load, I have date field DateCompleted.

Only for Insert

Thanks in Advance...

[jagan]

Hi,

First understand what is incremental load and how it works.  Check Qlikview help file there is very good explanations there with sample script.

Also go through the links shared by gwassenaar‌ and Kush141087‌ to understand.

The purpose of incremental load is to reduce the load on databases by reading only the latest records.

LET vToday = Date(Floor(LastReloadTime()));

LET vQVDPath = 'C:\QVD\';

// If QVD is found then reload after last reload time if not reload from Jan-2010  (Adjust this based on your requirement)

IF Alt(FileSize('$(vQVDPath)' & '\Data.qvd'), 0) > 0 THEN

  LET vLastReloadData = Date(Floor(ReloadTime()), 'MM/DD/YYYY');

ELSE

  LET vLastReloadData = Date(MakeDate(2010, 1), 'MM/DD/YYYY');

END IF

Example:

SELECT

*

FROM TableName

WHERE DateFieldName >= TO_DATE('$(vLastReloadData)', 'MM/DD/YYYY');

// Load Past data from QVD file

IF Alt(FileSize('$(vQVDPath)' & '\Data.qvd'), 0) > 0 THEN

  Concatenate(Actual)

  LOAD *

  FROM [$(vQVDPath)\Data.qvd] (qvd)

  WHERE DateFieldName < '$(vLastReloadTimeActual)';

END IF

STORE Data INTO [$(vQVDPath)\Data.qvd];

DROP TABLE Data;

//Now your QVD has all old and new data, you can use this qvd anywhere you required.

Regards,

Jagan.

[qv_testing]

Thanks Jagan..

I am Fetching from SQL Data,

I have implemented as above code..

After script execution i am getting failure.

Any idea??

[Gysbert_Wassenaar]

Posting the error might be an idea. Most of can't read minds.

[qv_testing]

HI,

Is this Correct ??

Data i am fetching from SQL....

LET vToday = Date(Floor(ReloadTime()));

LET vQVDPath = 'D:\QVD\';

IF Alt(FileSize('$(vQVDPath)' & '\Data.qvd'), 0) > 0 THEN

  LET vLastReloadData = Date(Floor(ReloadTime()), 'MM/DD/YYYY');

ELSE

  LET vLastReloadData = Date(MakeDate(2014, 1), 'MM/DD/YYYY');

END IF

Table:

SELECT

     Category ID,

    Category,

    Product,

    DateCompleted,

    Price

FROM Category

WHERE DateCompleted>= TO_DATE('$(vLastReloadData)', 'MM/DD/YYYY');

IF Alt(FileSize('$(vQVDPath)' & '\Data.qvd'), 0) > 0 THEN

  Concatenate(Table)

  LOAD *

  FROM [$(vQVDPath)\Data.qvd] (qvd)                        Note: 1st time i don't have this QVD

  WHERE DateCompleted< '$(vLastReloadData)';

END IF

STORE Data INTO [$(vQVDPath)\Data.qvd];

DROP TABLE Table;

Thanks..