How is data stored in Qlik Sense internally ?

[Anonymous]

We are currently using Qlik Sense with connections to Multiple Excel files and SQL Database tables. These are huge files with lot of data for analysis. How does data flow happen in Qlik Sense ? Does the entire tables get stored into Memory of Qlik Engine server while we open the app ? How are the data models stored in the backend ? Are QVD files involved in this data flow ? If yes, what will be the physical location of the QVD file.

The baseline question - is data (from external sources)stored in file system or memory ?

[danansell42]

Hi Pradeep

The below link is a good explanation of how qlik sense manages data.

This was also available as a webinar during Qlik Summer school.

http://qlikdevgroup.com/wp-content/uploads/Hamburg-Event01-Session01-QIX-Engine-Under-the-hood.pdf‌

Dan

[Anonymous]

Thanks for sharing the information Dan. It was very useful. I still need to understand while connecting to external databases, does Qlik store a local copy of the data in the QIX Internal Database or in Windows file system as QVD files ?

[danansell42]

The data would be loaded into the application.

If you want to store in QVD's for future reloading then you would need to state this in your load script by using a store statement.

[Anonymous]

When you say application, in the backend it just a QVF file. How does this data from external database gets stored inside it ? I do not think App file will contain the data.

Consider this scenario. I have an App in Qlik Sense and the data connection is through an OLEDB connection to MS-SQL server and the tables are around 3GB in size. Also, i make few changes in the Data Model and then load the data. Now when I open the App, the data is present is memory of the server and calculations happen and I see the visualization.

Question is -- When I close the app, what happens to the data in the memory from the database ? Does it get flushed away from memory or stored somewhere internally in the server ?

If it is stored in server, what is the location of those data files?

[danansell42 · Accepted solution]

When you reload an app (qvf file) the data is loading directly into this app and stored in a compressed format (using symbol and data tables with binary pointers - as per the document i gave the link too).

The below document is a good read to help understand how qlikview/qliksense store data in the cache

The QlikView Cache