Qlik Community

Third-Party Qlik Jobs

Area for people to learn more about current Qlik related job postings.

ANANNEBO
New Contributor

Incremental logic help

I need to implement incremental logic for the below data based on Cycle start data. If the previous Cycle start date data changes, in qlik how can we achieve that inserted or updated data. 

Example: 

ID, Name, Cycle Start Date

1,aa,1/2/2019

2,ee,1/3/2019

I am extracting data incrementally with where Cycle Start Date > max(Cycle Start date). But the challenge here is even the previous cycle start date data also modifying and i am not sure how to extract that modify data into qlik with incremental load. 

Please help me out to overcome this issue. 

Thanks,

Aamani

 

 

4 Replies
MVP
MVP

Re: Incremental logic help

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. if the dates in your data can change and you want to pick up that change, what other field can you use to distinguish the changed from the unchanged?

If you don't have some way of doing that, you will be unable to use incremental loading.

Logic will get you from a to b. Imagination will take you everywhere. - A Einstein
Partner
Partner

Re: Incremental logic help

try this

1st make your QVD

then

IF IsNull(QvdCreateTime('lib://........./example.qvd')) = -1 then
    example:
    LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;

    Store example into 'lib://...../QVD_example.qvd';

    Drop table example ;

//Incremental Load
else
    MaxModificationDate:
      LOAD
          Max("Cycle Start Date") as MaxModifDate
      FROM [lib://.../QVD_example.qvd]
      (qvd);

    LET vLastModificationDate = peek('MaxModifDate',0,'MaxModificationDate');

    DROP table MaxModificationDate;
    
    Example:
    LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;
    (qvd);

    Concatenate

    LOAD
     Id,
         "Cycle Start Date"
    FROM [lib://....] ;   
    where "Modification Date" > $(vLastModificationDate);

    Store OrderDetails into 'lib://...../QVD_example.qvd';

    Drop table Example;  
Endif

Then Check your QVD

 

Regrads

 

ANANNEBO
New Contributor

Re: Incremental logic help

Hi , 

Actually there is no such date change column in our back-end table like update timestamp or modify time stamp. is there any other way to do incremental.

aaaa

 

Highlighted
MVP
MVP

Re: Incremental logic help

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. 

If you don't have some way of doing that, you will be unable to use incremental loading.

Logic will get you from a to b. Imagination will take you everywhere. - A Einstein