QlikView use massive caching. This good and done by good design.
When QlikView starts to use more RAM that your Working Memory Set High % as configured in the QMC it will flush old things out of cache.
Hence this is not problem per se, merely QlikView doing goo in memory analytics.
I know this is a bit unnerving until you get used to it, but notwithstanding that is it causing any actual issues ?.
You are right, cache can only be cleared upon a service restart. However, QlikView server manages memory in an optimal way. Cached results are shared between users so there is no duplication within memory as such.
Here's a summary from QlikView's Server Memory tech brief:
- QlikView Server will cache all result sets whilst RAM available for allocation.
- QlikView Server will only release memory when unloading documents. When the application is unloaded from memory, the total amount of allocated memory will drop by the same amount as originally allocated by the application. However, please note that cached result sets will persist in memory as there is no reason to remove any cached result set that might be beneficial later on if there are no other requests for usage of the allocated memory.
- When the value of working set-min is reached, old sessions and cached results are purged to make room for the new values.
- The age, size and the time for calculation are factors in the prioritization of values to purge.
- QlikView Server will purge old sessions when the “maximum inactive session time” is reached.
- High memory usage of the QlikView Server is usually the result of many cached results and as long as paging does not occur, it is a good thing.
Hope that helps.
The cache is only populated by "query" results, things you select or search.
Optimization of QlikView docuemnts starts with not loading any columns you don't really need, reducing the number of discrete field values AKA cardinality (improves compression, symbol table size, and overall lookup speed) and the number of hops between data model extremities (press Ctrl-T in QV Desktop with your document loaded and count the number of association jumps).
But then again, a document with an inordinated amount of tables and a disgusting number of associative key values my perform decently on a huge number of fast cores with 1 Tb of RAM, while a simple 1 fact table-and-three-dimension-table document may perform in a sluggish way on a low-end server.. YMMV.
I wonder if you are getting confused between cache and general performance? QV Server will always take as much RAM as you allow it. So if you allow it 75% of your available RAM, it will always use all of that. You can reduce this by editing your Working Set settings.
You can certainly improve performance by optimising data structures. As of Best Practice, the rule of thumb would be use a link table for small data structures (eg dimensions around a large link table) and when performance starts to wane (as data volumes grow), then start consolidating into a single table. There's no point in prematurely optimising, you're better to go for simplicity and maintainability first.
There's obviously lots more to it and there's never a one size fits all but hopefully this is enough to answer your original question.
The behavior of the QlikView Server (QVS) memory manager can be summarized as follows:
- Once you get hold of "precious" memory bytes, never release them (a lot of rings in this world) You never know when you'll need them...
- Under duress, release memory until reaching the Lower Working set Limit (usually 70% of physicak RAM) Then STOP!
- The maximum amount of RAM you can ever use is the Working set High Limit (usually 90% of available RAM) Oh, leave some bits to poor Bill... (Windows), so never set this to 100% because your will run out of oxygen...
In reality, QlikView Sister (QVS) will not acknowledge QlikView brother (QDS = Qlik Distressed Son - or Slave -, the reload manager) and therefor they will fight for precious memory bytes to the death...May cause some family fights...
And the cache will not necessarily play an active part in this, because even when nobody is visiting the QlikView AP (and all document loads have timed-put), the memory will not be released. Ever.