Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
deep2021
Creator III
Creator III

Why is my document about 8 times bigger in memory than on disk?

Hi,

Although my app size is around 8 GB, my RAM footprint is showing around 8 times bigger than my app size.

 

deep2021_0-1668178019273.png

 

Can you please suggest the reason behind this and please suggest how to reduce the RAM footprint for this?

 

Thanks

2 Solutions

Accepted Solutions
mountaindude
Partner Ambassador
Partner Ambassador

This is perfectly normal - at least it can easily happen given som datasets and how that data is modelled in the app.

I researched this some years ago and managed to create apps that expanded a factor of 70-80 (and more in some cases) when loaded into RAM. 
Admittedly the data loaded into those apps was tailored to be small on disk and expand massively when loaded into memory, but that experiment proved the point that size on disk and in memory can be two very different things.

The technical reason why this happen is slightly complex (this is is really the core of Qlik's secret sauce - what makes the Sense and QlikView experience so unique), but high cardinality data is usually a very contributing factor.
I.e. fields in the data that contain lots of different, unique values that cannot be indexed in a space-efficient manner.

If you have many such fields in the dataset you are likely to see a "memory explosion" when the app is loaded from disk.

The way I usually tackle these things is to ask myself if all data in the app really needs to be indexed.

For example, maybe only some dimensional fields need to be indexed, while those high cardinality facts/measurement  fields don't need to be indexed. There is usually no (or little) need to be able to do full text searching on those fields. 

 

 

Please mark the post as a solution if it provided you with a solution to the topic at hand. Thanks!

View solution in original post

Chip_Matejowsky
Support
Support

Hi @deep2021,

@mountaindude is spot on with his answer. Qlik Knowledgebase article How much RAM is consumed per application? does a pretty good job in explaining this phenomenon.

 

Best Regards

Principal Technical Support Engineer with Qlik Support
Help users find answers! Don't forget to mark a solution that worked for you!

View solution in original post

2 Replies
mountaindude
Partner Ambassador
Partner Ambassador

This is perfectly normal - at least it can easily happen given som datasets and how that data is modelled in the app.

I researched this some years ago and managed to create apps that expanded a factor of 70-80 (and more in some cases) when loaded into RAM. 
Admittedly the data loaded into those apps was tailored to be small on disk and expand massively when loaded into memory, but that experiment proved the point that size on disk and in memory can be two very different things.

The technical reason why this happen is slightly complex (this is is really the core of Qlik's secret sauce - what makes the Sense and QlikView experience so unique), but high cardinality data is usually a very contributing factor.
I.e. fields in the data that contain lots of different, unique values that cannot be indexed in a space-efficient manner.

If you have many such fields in the dataset you are likely to see a "memory explosion" when the app is loaded from disk.

The way I usually tackle these things is to ask myself if all data in the app really needs to be indexed.

For example, maybe only some dimensional fields need to be indexed, while those high cardinality facts/measurement  fields don't need to be indexed. There is usually no (or little) need to be able to do full text searching on those fields. 

 

 

Please mark the post as a solution if it provided you with a solution to the topic at hand. Thanks!
Chip_Matejowsky
Support
Support

Hi @deep2021,

@mountaindude is spot on with his answer. Qlik Knowledgebase article How much RAM is consumed per application? does a pretty good job in explaining this phenomenon.

 

Best Regards

Principal Technical Support Engineer with Qlik Support
Help users find answers! Don't forget to mark a solution that worked for you!