topic Re: Incremental Load (MERGE) using NON-QVD method. in App Development

Incremental Load (MERGE) using NON-QVD method.

venkatramachandran — Wed, 12 Feb 2025 21:32:21 GMT

Is there a way to do Incremental Load using NON-QVD method?

I have a source table. dw.vr_sample_table. It has columns such as a sample_id int, name varchar and loaddate timestamp.

Is it possible to do using Partial-Reload? Please help to create the script. Thank you

Re: Incremental Load (MERGE) using NON-QVD method.

Vegar — Wed, 12 Feb 2025 22:29:02 GMT

If I understand you correctly you want to use merge load without storing output to qvd. Yes you can use merge load into an app from any load/select script (do not need to be qvd)

In order to just load the delta changes you would need to always do a partial load (not full load) when reloading the app.

Please take a look at this page: https://help.qlik.com/en-US/sense/November2024/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptPrefixes/Merge.htm

Re: Incremental Load (MERGE) using NON-QVD method.

rwunderlich — Wed, 12 Feb 2025 22:53:11 GMT

Incremental reload is generally understood to mean loading rows that exist in Set1 (the DB table) but do not exist in Set2 (the QVD).

In your case you refer to only one Set, the db table. What defines incremental or partial load for you? Do you want to load rows from a point in time? What defines that time?

-Rob

Re: Incremental Load (MERGE) using NON-QVD method.

venkatramachandran — Thu, 13 Feb 2025 04:29:41 GMT

1. Uncheck the "Partial Laod" in QMC
2. Run LOAD statement from set 1 where data > '1900-01-01' (Initial Full Load). Now, data can be previwed in "Data Model Viewer", Consider this data in memory as set 2. After that, Set the lastrundate = now()
3. Check the "Partial Load"
4. Run ADD Only LOAD statement from set 1 where data > 'lastrundate' (this is next run)
5. Full Load + Additional Data can be seen in 'Data Model Viewer'. I believe it is in memory.
6. Instead of using ADD Only (step 4), is it possible to use 'MERGE' ? The merge should do merge at the memory level. We are not using xlxs or table for set2. Here, set2 is model memory

Re: Incremental Load (MERGE) using NON-QVD method.

venkatramachandran — Thu, 13 Feb 2025 14:47:22 GMT

Below is the question..I followed the steps. How to do MERGE Only LOAD into Data Model memory table.

1. Uncheck the "Partial Load" in QMC
2. Run LOAD statement from set 1 where data > '1900-01-01' (Initial Full Load). Now, data can be previwed in "Data Model Viewer", Consider this data in memory as set 2. After that, Set the lastrundate = now()
3. Check the "Partial Load"
4. Run ADD Only LOAD statement from set 1 where data > 'lastrundate' (this is next run)
5. Full Load + Additional Data can be seen in 'Data Model Viewer'. I believe it is in memory.
6. Instead of using ADD Only (step 4), is it possible to use 'MERGE' ? The merge should do merge at the memory level. We are not using xlxs or table for set2. Here, set2 is model memory answer

Re: Incremental Load (MERGE) using NON-QVD method.

venkatramachandran — Thu, 13 Feb 2025 22:53:37 GMT

Below is the question..I followed the steps. How to do MERGE Only LOAD into Data Model memory table.