Qlik Community

Connectivity & Data Prep

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Announcements
Don't miss the upcoming Q&A with Qlik session on Qlik Application Automation on November 16th! REGISTER NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Sansraison
Contributor II
Contributor II

Incremental Load from Hive DataBase

Hello Everyone !

I'm new on Qlik and well... i am stuck and i'm already sorry for my English (French here..)

I want to do an incremental data load (Insert, Update, & Delete) process in Qlik Sense, then store the data into a qvd file where the source of data is from a Hive DataBase.

Here's a script exemple from a section of my app where i would like to perform an incremental load :

    [routage_cibles_c]:
    NoConcatenate

    SQL SELECT *
    FROM HIVE.$(database).$(routage_cibles_c);

    Store [routage_cibles_c] into [lib://$(QVDOutRepo)/$(QVDOutType)_$(QVDOutSource1)_routage_cibles_c.qvd];

    Drop Table routage_cibles_c;

I have found a lot of ressources with the use of QVD files, but not directly with a connection to a database.

The script found on Qlik Help which i'd like to force into my code use a TimeStamp mechanic which i can't apply (i think) on my use case, right ?

    Let ThisExecTime = Now( );

    QV_Table:

    SQL SELECT PrimaryKey, X, Y FROM DB_TABLE

    WHERE ModificationTime >= #$(LastExecTime)#

    AND ModificationTime < #$(ThisExecTime)#;

 

    Concatenate LOAD PrimaryKey, X, Y FROM File.QVD

    WHERE NOT EXISTS(PrimaryKey);

 

    Inner Join SQL SELECT PrimaryKey FROM DB_TABLE;

 

    If ScriptErrorCount = 0 then

    STORE QV_Table INTO File.QVD;

    Let LastExecTime = ThisExecTime;

    End If

Can anyone explain to me how this works, how can this be merge into my code and what are the best case practice to use ?

Thanks a lot !

Wishing you a good day !

0 Replies