Incremental load

[vadim_grab]

Hi, All.

I'm studying examples scenario of incremental data loading to use best practice in current project.

Most common scenarion is Insert, Update and Delete.

One of the examples below (from book Mastering Qlik Sense Packt March 2018).

I have a question -- What will be if COUNTRY don't have a new data? 

The table COUNTRY will not be created, so the data will not be loaded at all.

This question also actuals  for most of the incremental load examples that can be found in Qlik community.

Maybe I'm missing something.

Many thanks for your comments and advices!

Best regards,

Vadim.

//I check if the file exists. If it doesn't I do a full load, if it does I perform the incremental load.

IF IsNull(QvdCreateTime('C:\myQVD\Countries.qvd')) THEN

TRACE Performing full load of data source;

Countries:

SELECT * FROM COUNTRY;

STORE Countries INTO C:\myQVD\Countries.qvd (qvd);

ELSE

TRACE File exists. Performing incremental load;

maxDate:

LOAD max(LastModifiedDate) AS MaxModifiedDate

FROM c:\myQVD\Countries.qvd (qvd);

TRACE Storing the most current modified date into a variable.;

LET vMaxModifiedDate = peek(MaxModifiedDate);

TRACE Loading new records from the source.;

Countries:

SELECT * FROM COUNTRY

WHERE LastModifiedDate > $(vMaxModifiedDate)     // My question - What will be if COUNTRY don't have a new data

;

TRACE Appending stored records to  new ones.;

Concatenate

LOAD * FROM c:\myQVD\Countries.qvd (Qvd)

WHERE Not EXISTS(%Id);

TRACE We join the final table with all the current keys in the source to remove the delete records.;

INNER JOIN

SQL SELECT %Id FROM COUNTRY;

TRACE Storing modified qvd file.;

STORE Countries INTO c:\myQVD\Countries.qvd (Qvd);

ENDIF

[andrey_krylov]

In this case you get the table with all fields and no rows but after concotenating

Concatenate

LOAD * FROM c:\myQVD\Countries.qvd (Qvd)

WHERE Not EXISTS(%Id);

you get the full table

[mohan_1105]

Hi Vadim,

If I understood correctly, your query will not fetch any new records and the qvd contains only the records fetched in the previously loaded data.

[marcus_sommer]

Something like:

t: load F1, F2, ... from Source where 1=2;

will lead to an empty table - all fields are available but no records. And this meant also that you could concatenate the qvd to such a table.

- Marcus