Partial Reload not working as expected

[profilejamesbond]

Hi,

I implemented partial reload in Qlik but I see it is not working as expected.

Scenario:

I want to load full table on first of each month and rest of the month days, I want to load them partially.

My code:

//full reload

If not IsPartialReload() then

table:

load * from sql.datasource;

end if

//partial reload

If IsPartialReload() then

table_tmp:

NoConcatenate

replace load * from table where date<date(today()-5, 'DD.MM.YYYY');

Concatenate(table_tmp)

add load * from sql.datasource where datedate>=date(today()-5, 'DD.MM.YYYY');

end if

The problem is, it is always loading and replacing the existing table with last 5 days data such as this line and concatenating with the other data. Seems to me only this line is working:

add load * from sql.datasource where datedate>=date(today()-5, 'DD.MM.YYYY');

How to fix this problem, I need old data plus newly available data.

Thanks

[diegozecchini]

Hi
The issue with your Qlik script lies in how you are handling the partial reload. Specifically, the "replace" keyword in "replace load" removes the existing table_tmp each time the script runs.
After this, you're adding only the last 5 days data, effectively overwriting the previously loaded data.

To achieve your goal loading the full table on the first of the month and only appending new data on subsequent days you need to properly manage the handling of the old data. Here's a revised script:

// Full reload: Load the full table on the first of each month
If not IsPartialReload() then
table:
load * from sql.datasource;
end if;

// Partial reload: Append data only for the last 5 days
If IsPartialReload() then
// Load existing data (to preserve old records)
table_tmp:
load * resident table;

// Append new data for the last 5 days
add load * from sql.datasource where date >= date(today()-5, 'DD.MM.YYYY');
end if;

[profilejamesbond]

Hi @diegozecchini,

Following load will not work in the section: IsPartialReload()

table_tmp:
load * resident table;

https://help.qlik.com/en-US/sense/May2024/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptPrefixes/...

It needs something like add / replace / etc...

[diegozecchini]

you are right, my mistake.

try this:

If IsPartialReload() then
// Preserve existing data by using ADD
add load * resident table;

// Append new data from SQL source
add load * from sql.datasource where date >= date(today()-5, 'DD.MM.YYYY');
end if;