Qlik Community

Connectivity & Data Prep

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Announcements
Support Case Portal has moved to Qlik Community! Read the FAQs to start exploring Support resources.
cancel
Showing results for 
Search instead for 
Did you mean: 
Sansraison
Contributor II
Contributor II

Incremental Load from Hive DataBase

Hello Everyone !

I'm new on Qlik and well... i am stuck and i'm already sorry for my English (French here..)

I want to do an incremental data load (Insert, Update, & Delete) process in Qlik Sense, then store the data into a qvd file where the source of data is from a Hive DataBase.

Here's a script exemple from a section of my app where i would like to perform an incremental load :

    [routage_cibles_c]:
    NoConcatenate

    SQL SELECT *
    FROM HIVE.$(database).$(routage_cibles_c);

    Store [routage_cibles_c] into [lib://$(QVDOutRepo)/$(QVDOutType)_$(QVDOutSource1)_routage_cibles_c.qvd];

    Drop Table routage_cibles_c;

I have found a lot of ressources with the use of QVD files, but not directly with a connection to a database.

The script found on Qlik Help which i'd like to force into my code use a TimeStamp mechanic which i can't apply (i think) on my use case, right ?

    Let ThisExecTime = Now( );

    QV_Table:

    SQL SELECT PrimaryKey, X, Y FROM DB_TABLE

    WHERE ModificationTime >= #$(LastExecTime)#

    AND ModificationTime < #$(ThisExecTime)#;

 

    Concatenate LOAD PrimaryKey, X, Y FROM File.QVD

    WHERE NOT EXISTS(PrimaryKey);

 

    Inner Join SQL SELECT PrimaryKey FROM DB_TABLE;

 

    If ScriptErrorCount = 0 then

    STORE QV_Table INTO File.QVD;

    Let LastExecTime = ThisExecTime;

    End If

Can anyone explain to me how this works, how can this be merge into my code and what are the best case practice to use ?

Thanks a lot !

Wishing you a good day !

0 Replies