Data storage in QVF vs QVD in Qlik Sense

[Anonymous]

I have been doing an experiment to isolate the Data layer in Qlik Sense from the Application. I also read about the associative engine and in memory technologies that make Qlik Sense so responsive. Now the confusing part.

- QVD is a buffer that already organizes data that can be consumed by QIX and can be scheduled for reload.

Then, why do Applications need to persist data to the QVF file if it is a 'in memory' technology. Or is the QVF behaviour different in Qlik Sense Desktop and Qlik Sense Enterprise Hub / Qlik Analytics Platform?

Is there a way to configure the Qlik applications that contain charts, visualizations (ie the presentation and logic layer) so that they do not save any data? Replicate a behaviour similar to Direct Discovery perhaps, so that the apps only contain logic, data load script and visualizations. No data.

The main problem I am trying to solve comes because of the shared folder model where apps belonging to multiple streams end up in the same folder (and meaning the same disks). However the QVD files can be saved to different folders by creating a separate connection for each folder (at least in the Desktop edition).

Questions:

- How do I create pure in memory apps without data persistence to files?

- Are there ways to segment apps belonging to separate streams in their own folders?

- What other ways exist to get physical separation to match logical separation in a multi tenant deployment?

Kind regards,

Ani

[Anonymous · Accepted solution]

I like QVD files and think they are good :

• A QVD generator can generate a QVD
  • This is generated & stored once, thus by definition giving a single version of truth of the data it contains

• QVD's form the basis of incremental data loading

• QVD's can [should] be loaded into qvf Apps using an optimised which is fast & efficient

• Thus a single QVD can be used to populate multiple qvf Apps
  • Thus eliminating the need for multiple full loads from the data sources
  • And benefit the qvf Apps as they load from a single version of the truth

[Anonymous]

Thank you Bill. Do you also have a view on the remaining questions? I did perform a simple experiment wherein the Application had a data load script that stored data in QVD and I created a simple table element with a few dimensions and a measure.

I was hoping that there would be partitioning of data and application logic between the QVF and QVD file. But it seems the QVF file continues to hold data. (I opened the app on another machine and it had data embedded in it).

So I now understand that the purpose of QVD is of course along the points you mention. Velocity and veracity. Like you said, one version of the truth.

[Anonymous]

Yup - That is how Qlik works :

• The qvf is loaded with data
• The qvf is stored to disk with this data in it
• The qvf metadata and application is then loaded into RAM
• When the qvf is used by users only what is in RAM is used and what is on disc is never accessed

Quite sensible this, as otherwise if the Qlik Sense Server / Desktop is restarted the users would have no data to look at until everything is reloaded again.  In a large Server environment all these reloads could take quite some hours which would be an unacceptable data non availability after a simple restart.