5 Replies Latest reply: Jul 29, 2016 2:07 PM by hoang vo RSS

    How to query only data not in a qvd

    hoang vo

      Hi,

       

      Is it possible to do an SQL query for data that does not exist in a QVD.

       

      I have a table with 2 columns, acct and desc.

      both exist in a qvd file.

       

      i've loaded the file into a table [accnt]

      but i want to do a select statement based on the qlik [accnt] table created earlier, and not sure how to approach this.

      also ,  i dont want to do a fetch on the sql table into qlik memory, only fetch records that doesnt exist in the qvd into memory.

        • Re: How to query only data not in a qvd
          Harvey Johal

          You may want to think about a strategy where you pull data from your SQL database into a QVD and performing incremental loads.  The loads from QVD will be faster once loading into the app.

           

          http://help.qlik.com/en-US/sense/3.0/Subsystems/Hub/Content/LoadData/use-QVD-files-incremental-load.htm

           

          A Good example of a script is within the SalesForce.QVW sample application for QlikView: 

          Qlik Demos: See QlikView in Action | Demo.Qlik.Com

           

          This sample shows script that checks a reloadhistory to see when to append data from SQL into a .QVD and to load data from the QVD.  Take a look at how it's built in QlikView - you can test it there and/or test and apply the logic within Qlik Sense.


          Please mark the appropriate replies as CORRECT / HELPFUL so our team and other members know that your question(s) has been answered to your satisfaction.

            • Re: How to query only data not in a qvd
              hoang vo

              The help link i have already reviewed before posting the question, i actually have the incremental load (update/delete/insert) apply to etl process.  The data for this , had a date to query against so it was simple enough to implement the incremental load.

               

              The data I am working on now is just a table and I am trying to create the QVD that will be consume by the application. The QVD creation process currently download and reload the whole table, but that takes too long as there are too many records and we have to connect to the the source to download it. The source is external to the qliksense host.

               

              So my predicament is that, how do I use the QVD file that was downloaded initially, and query the database for only those that does not exists in the qvd if possible.

            • Re: How to query only data not in a qvd
              Rob Wunderlich

              Is there some sort of incrementing value in the already loaded records, like a id number or a modification date?

               

              -Rob

              • Re: How to query only data not in a qvd
                hoang vo

                There are no incremental values, just unique value(account).

                There are no dates either.

                 

                The table only has new records provided that the record has the unique account.

                 

                The only thing i can think of is to pull all of the unique value and write it to a csv, and then run the sql query against the csv for all records not exists() in it.