You may want to think about a strategy where you pull data from your SQL database into a QVD and performing incremental loads. The loads from QVD will be faster once loading into the app.
A Good example of a script is within the SalesForce.QVW sample application for QlikView:
This sample shows script that checks a reloadhistory to see when to append data from SQL into a .QVD and to load data from the QVD. Take a look at how it's built in QlikView - you can test it there and/or test and apply the logic within Qlik Sense.
Please mark the appropriate replies as CORRECT / HELPFUL so our team and other members know that your question(s) has been answered to your satisfaction.
The help link i have already reviewed before posting the question, i actually have the incremental load (update/delete/insert) apply to etl process. The data for this , had a date to query against so it was simple enough to implement the incremental load.
The data I am working on now is just a table and I am trying to create the QVD that will be consume by the application. The QVD creation process currently download and reload the whole table, but that takes too long as there are too many records and we have to connect to the the source to download it. The source is external to the qliksense host.
So my predicament is that, how do I use the QVD file that was downloaded initially, and query the database for only those that does not exists in the qvd if possible.
There are no incremental values, just unique value(account).
There are no dates either.
The table only has new records provided that the record has the unique account.
The only thing i can think of is to pull all of the unique value and write it to a csv, and then run the sql query against the csv for all records not exists() in it.