Incremental logic help

[ANANNEBO]

I need to implement incremental logic for the below data based on Cycle start data. If the previous Cycle start date data changes, in qlik how can we achieve that inserted or updated data. 

Example: 

ID, Name, Cycle Start Date

1,aa,1/2/2019

2,ee,1/3/2019

I am extracting data incrementally with where Cycle Start Date > max(Cycle Start date). But the challenge here is even the previous cycle start date data also modifying and i am not sure how to extract that modify data into qlik with incremental load. 

Please help me out to overcome this issue. 

Thanks,

Aamani

[jonathandienst]

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. if the dates in your data can change and you want to pick up that change, what other field can you use to distinguish the changed from the unchanged?

If you don't have some way of doing that, you will be unable to use incremental loading.

[taufiqmaulanaa]

try this

1st make your QVD

then

IF IsNull(QvdCreateTime('lib://........./example.qvd')) = -1 then
    example:
    LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;

    Store example into 'lib://...../QVD_example.qvd';

    Drop table example ;

//Incremental Load
else
    MaxModificationDate:
      LOAD
          Max("Cycle Start Date") as MaxModifDate
      FROM [lib://.../QVD_example.qvd]
      (qvd);

    LET vLastModificationDate = peek('MaxModifDate',0,'MaxModificationDate');

    DROP table MaxModificationDate;
    
    Example:
    LOAD
        Id,
         "Cycle Start Date"
    FROM [lib://....] ;
    (qvd);

    Concatenate

    LOAD
     Id,
         "Cycle Start Date"
    FROM [lib://....] ;   
    where "Modification Date" > $(vLastModificationDate);

    Store OrderDetails into 'lib://...../QVD_example.qvd';

    Drop table Example;  
Endif

Then Check your QVD

Regrads

[ANANNEBO]

Hi , 

Actually there is no such date change column in our back-end table like update timestamp or modify time stamp. is there any other way to do incremental.

aaaa

[jonathandienst]

You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. 

If you don't have some way of doing that, you will be unable to use incremental loading.

[Brett_Bleess]

Just wanted to add a link to the Help on this topic in case that may provide some further assistance with things for future visitors to the post.

https://help.qlik.com/en-US/qlikview/April2019/Subsystems/Client/Content/QV_QlikView/QVD_Incremental...

Regards,
Brett