topic Re: Incremental logic help in QlikView

Incremental logic help

ANANNEBO — Wed, 25 Nov 2020 16:16:04 GMT

I need to implement incremental logic for the below data based on Cycle start data. If the previous Cycle start date data changes, in qlik how can we achieve that inserted or updated data.

Example:

ID, Name, Cycle Start Date

1,aa,1/2/2019

2,ee,1/3/2019

I am extracting data incrementally with where Cycle Start Date > max(Cycle Start date). But the challenge here is even the previous cycle start date data also modifying and i am not sure how to extract that modify data into qlik with incremental load.

Please help me out to overcome this issue.

Thanks,

Aamani

Re: Incremental logic help

jonathandienst — Fri, 25 Jan 2019 05:33:15 GMT

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. if the dates in your data can change and you want to pick up that change, what other field can you use to distinguish the changed from the unchanged?

If you don't have some way of doing that, you will be unable to use incremental loading.

Re: Incremental logic help

taufiqmaulanaa — Fri, 25 Jan 2019 11:32:47 GMT

try this

1st make your QVD

then

IF IsNull(QvdCreateTime('lib://........./example.qvd')) = -1 then
    example:
   LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;

   Store example into 'lib://...../QVD_example.qvd';

   Drop table example ;

//Incremental Load
else
   MaxModificationDate:
   LOAD
         Max("Cycle Start Date") as MaxModifDate
    FROM [lib://.../QVD_example.qvd]
   (qvd);

   LET vLastModificationDate = peek('MaxModifDate',0,'MaxModificationDate');

   DROP table MaxModificationDate;

    Example:
    LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;
    (qvd);

   Concatenate

    LOAD
     Id,
         "Cycle Start Date"
    FROM [lib://....] ;
    where "Modification Date" > $(vLastModificationDate);

    Store OrderDetails into 'lib://...../QVD_example.qvd';

    Drop table Example;
Endif

Then Check your QVD

Regrads

Re: Incremental logic help

ANANNEBO — Fri, 25 Jan 2019 13:26:15 GMT

Hi ,

Actually there is no such date change column in our back-end table like update timestamp or modify time stamp. is there any other way to do incremental.

aaaa

Re: Incremental logic help

jonathandienst — Tue, 29 Jan 2019 12:57:19 GMT

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source.

If you don't have some way of doing that, you will be unable to use incremental loading.

Re: Incremental logic help

Brett_Bleess — Tue, 14 Apr 2020 17:32:24 GMT

Just wanted to add a link to the Help on this topic in case that may provide some further assistance with things for future visitors to the post.

https://help.qlik.com/en-US/qlikview/April2019/Subsystems/Client/Content/QV_QlikView/QVD_Incremental.htm

Regards,
Brett