Incremental Load from Hive DataBase

Sansraison — Wed, 21 Apr 2021 10:00:21 GMT

Hello Everyone !

I'm new on Qlik and well... i am stuck and i'm already sorry for my English (French here..)

I want to do an incremental data load (Insert, Update, & Delete) process in Qlik Sense, then store the data into a qvd file where the source of data is from a Hive DataBase.

Here's a script exemple from a section of my app where i would like to perform an incremental load :

[routage_cibles_c]:
NoConcatenate

SQL SELECT *
FROM HIVE.$(database).$(routage_cibles_c);

Store [routage_cibles_c] into [lib://$(QVDOutRepo)/$(QVDOutType)_$(QVDOutSource1)_routage_cibles_c.qvd];

Drop Table routage_cibles_c;

I have found a lot of ressources with the use of QVD files, but not directly with a connection to a database.

The script found on Qlik Help which i'd like to force into my code use a TimeStamp mechanic which i can't apply (i think) on my use case, right ?

Let ThisExecTime = Now( );

QV_Table:

SQL SELECT PrimaryKey, X, Y FROM DB_TABLE

WHERE ModificationTime >= #$(LastExecTime)#

AND ModificationTime < #$(ThisExecTime)#;

Concatenate LOAD PrimaryKey, X, Y FROM File.QVD

WHERE NOT EXISTS(PrimaryKey);

Inner Join SQL SELECT PrimaryKey FROM DB_TABLE;

If ScriptErrorCount = 0 then

STORE QV_Table INTO File.QVD;

Let LastExecTime = ThisExecTime;

End If

Can anyone explain to me how this works, how can this be merge into my code and what are the best case practice to use ?

Thanks a lot !

Wishing you a good day !

topic Incremental Load from Hive DataBase in Connectivity & Data Prep

Incremental Load from Hive DataBase