Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I need to implement incremental logic for the below data based on Cycle start data. If the previous Cycle start date data changes, in qlik how can we achieve that inserted or updated data.
Example:
ID, Name, Cycle Start Date
1,aa,1/2/2019
2,ee,1/3/2019
I am extracting data incrementally with where Cycle Start Date > max(Cycle Start date). But the challenge here is even the previous cycle start date data also modifying and i am not sure how to extract that modify data into qlik with incremental load.
Please help me out to overcome this issue.
Thanks,
Aamani
You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source. if the dates in your data can change and you want to pick up that change, what other field can you use to distinguish the changed from the unchanged?
If you don't have some way of doing that, you will be unable to use incremental loading.
try this
1st make your QVD
then
IF IsNull(QvdCreateTime('lib://........./example.qvd')) = -1 then
example:
LOAD
Id,
"Cycle Start Date"
FROM [lib://....] ;
Store example into 'lib://...../QVD_example.qvd';
Drop table example ;
//Incremental Load
else
MaxModificationDate:
LOAD
Max("Cycle Start Date") as MaxModifDate
FROM [lib://.../QVD_example.qvd]
(qvd);
LET vLastModificationDate = peek('MaxModifDate',0,'MaxModificationDate');
DROP table MaxModificationDate;
Example:
LOAD
Id,
"Cycle Start Date"
FROM [lib://....] ;
(qvd);
Concatenate
LOAD
Id,
"Cycle Start Date"
FROM [lib://....] ;
where "Modification Date" > $(vLastModificationDate);
Store OrderDetails into 'lib://...../QVD_example.qvd';
Drop table Example;
Endif
Then Check your QVD
Regrads
Hi ,
Actually there is no such date change column in our back-end table like update timestamp or modify time stamp. is there any other way to do incremental.
aaaa
You need some way to separate the data which is no longer changing (so can be read from your QVD) and data that needs to be read (or re-read) from the data source.
If you don't have some way of doing that, you will be unable to use incremental loading.
Just wanted to add a link to the Help on this topic in case that may provide some further assistance with things for future visitors to the post.
Regards,
Brett