topic Re: Loading data from SQL vs QVD, what's better and why? in QlikView

Loading data from SQL vs QVD, what's better and why?

Tue, 26 Jan 2016 16:03:37 GMT

Trying to implement an app with lots of SQL dependencies existing in DW. Not sure what would be better, load all the data/tables from

SQL using connection string or from QVD files?

thanks

Re: Loading data from SQL vs QVD, what's better and why?

Anonymous — Tue, 26 Jan 2016 18:31:46 GMT

The answer is the classic "it all depends...."

Do you already have a collection of pre-created QVD's to use ?

Re: Loading data from SQL vs QVD, what's better and why?

rwunderlich — Tue, 26 Jan 2016 18:35:09 GMT

I think it's almost always better to decouple the data extract and transformation from loading data in the final app. So in this case, I would create a layer of script to load from DW and store in QVDs. And then load the QVDs into the final app.

-Rob

Re: Loading data from SQL vs QVD, what's better and why?

maxgro — Tue, 26 Jan 2016 18:38:38 GMT

One of the things to take into account is the data volume; if you have "big" tables,

with qvd you can use an incremental load and reduce the time to get the data from the database (sql).

For "small" tables it's easier to always extract all the data from the database (sql).

Re: Loading data from SQL vs QVD, what's better and why?

Tue, 26 Jan 2016 21:33:41 GMT

No, don't have QVD files, tables are below 200K rows but data used on the application is comprised of 10 to 15 tabs/containers using data from 15 to 20 SQL tables, some tables have 5K rows, a few have around 150K rows.

Re: Loading data from SQL vs QVD, what's better and why?

Tue, 26 Jan 2016 21:34:48 GMT

No, big tables in my view, most tables are below 20K rows but, application is comprised of 10 to 15 tabs/containers using data from 15 to 20 SQL tables, some tables have 5K rows, a few have around 150K rows.

Re: Loading data from SQL vs QVD, what's better and why?

mmarchese — Mon, 20 Apr 2020 16:42:05 GMT

@rwunderlich wrote:
I think it's almost always better to decouple the data extract and transformation from loading data in the final app. So in this case, I would create a layer of script to load from DW and store in QVDs. And then load the QVDs into the final app.

I'm curious, can you explain your reasoning? Why do you "almost always" recommend this approach? I'm not using qvds much yet, and I'm wondering if I'm heading down a bad path.