Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Discover how organizations are unlocking new revenue streams: Watch here
cancel
Showing results for 
Search instead for 
Did you mean: 
venkatramachandran
Contributor II
Contributor II

Incremental Load (MERGE) using NON-QVD method.

Is there a way to do Incremental Load using NON-QVD method?

I have a source table. dw.vr_sample_table. It has columns such as a sample_id int, name varchar and loaddate timestamp.

Is it possible to do using Partial-Reload? Please help to create the script. Thank you

Labels (2)
5 Replies
Vegar
MVP
MVP

If I understand you correctly you want to use merge load without storing output to qvd.  Yes you can use merge load into an app from  any load/select script (do not need to be qvd)

 

In order to just load the delta changes you would need to always do a partial load (not full load) when reloading the app.

 

Please take a look at this page: https://help.qlik.com/en-US/sense/November2024/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptPref...

rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Incremental reload is generally understood to mean loading rows that exist in Set1 (the DB table) but do not exist in Set2 (the QVD). 

In your case you refer to only one Set, the db table. What defines incremental or partial load for you? Do you want to load rows from a point in time? What defines that time?

-Rob

venkatramachandran
Contributor II
Contributor II
Author

1. Uncheck the "Partial Laod" in QMC
2. Run LOAD statement from set 1 where data > '1900-01-01' (Initial Full Load). Now, data can be previwed in "Data Model Viewer",  Consider this data in memory as set 2. After that, Set the lastrundate = now()
3. Check the "Partial Load"
4. Run ADD Only LOAD statement from set 1 where data > 'lastrundate' (this is next run)
5. Full Load + Additional Data can be seen in 'Data Model Viewer'. I believe it is in memory.
6. Instead of using ADD Only (step 4), is it possible to use 'MERGE' ? The merge should do merge at the memory level. We are not using xlxs or table for set2. Here, set2 is model memory

venkatramachandran
Contributor II
Contributor II
Author

Below is the question..I followed the steps. How to do MERGE Only LOAD into Data Model memory table.

1. Uncheck the "Partial Load" in QMC
2. Run LOAD statement from set 1 where data > '1900-01-01' (Initial Full Load). Now, data can be previwed in "Data Model Viewer",  Consider this data in memory as set 2. After that, Set the lastrundate = now()
3. Check the "Partial Load"
4. Run ADD Only LOAD statement from set 1 where data > 'lastrundate' (this is next run)
5. Full Load + Additional Data can be seen in 'Data Model Viewer'. I believe it is in memory.
6. Instead of using ADD Only (step 4), is it possible to use 'MERGE' ? The merge should do merge at the memory level. We are not using xlxs or table for set2. Here, set2 is model memory answer

 

 
venkatramachandran
Contributor II
Contributor II
Author

Below is the question..I followed the steps. How to do MERGE Only LOAD into Data Model memory table.

1. Uncheck the "Partial Load" in QMC
2. Run LOAD statement from set 1 where data > '1900-01-01' (Initial Full Load). Now, data can be previwed in "Data Model Viewer",  Consider this data in memory as set 2. After that, Set the lastrundate = now()
3. Check the "Partial Load"
4. Run ADD Only LOAD statement from set 1 where data > 'lastrundate' (this is next run)
5. Full Load + Additional Data can be seen in 'Data Model Viewer'. I believe it is in memory.
6. Instead of using ADD Only (step 4), is it possible to use 'MERGE' ? The merge should do merge at the memory level. We are not using xlxs or table for set2. Here, set2 is model memory answer