topic Re: RAM usage increases when making selection in QlikView

RAM usage increases when making selection

Messen — Fri, 13 Aug 2021 08:55:22 GMT

Hi
I have a fairly large QlikView app. Around 85 mill. lines in Facttable. Uses about 15 GB of RAM when opened.

My Straight table has no calculated dimension and 1 expression = '1' for showing the lines.

So no calculations at all. Just showing lines. Very simple.

When making a selection in one field it takes about a minut for showing the data. The RAM usage of the app increases to more than double up - 35 GB. When data showes up in the table - RAM decreases again to 15 GB.

It doesn't matter whether my selection returns 1 or e.g. 50 lines in the table and doesn't matter whether I'm using Desktop or QlikView Server. Version are 12.10. (I know it is outdated....;-)

Tried different Datamodels : Star with 3 dimensions and all in one Fact table. Which surprisingly enough is faster than the model with 3 dimensions.

I have 25 fields. One field is a text field with average lenght of 100 and max lenght of 1000.

Anyone with suggestions why QlikView has to "unpack" so much data in the RAM for making a simpel selection and best of all - Suggestions for speeding up the time for the app.

Thank you in advance for every input 🙂

Regards

Sune

Re: RAM usage increases when making selection

marcus_sommer — Fri, 13 Aug 2021 11:18:14 GMT

The behaviour you noticed is called caching and is one of the main-features of Qlik - means most of calculation-results from selections are kept within the RAM and if this selection comes again it won't be new calculated else the cached results will be used. In many scenarios it's a real benefit - but it depends on the underlying data and the kind of analysis.

In your case Qlik tries to cache but discard it immediately because it hits any restriction which I don't know in detail. More background could you find here: The QlikView Cache - Qlik Community - 1468270

Beside this it's useful to limit the rows of large tables with a visibility-condition like:

count(distinct Dim1&Dim2&Dim3) < 10000

Also helpful should be to switch from the straight-table to a table-box which is AFAIK significantly faster as the table-charts.

Further I suggest to look on the cardinality of your fields especially for your mentioned text-field. The RAM consumption is directly related to the max. number of distinct field-values and you may be able to split the field and/or remove not really important information. The logic behind it is here very well explained:

The Importance Of Being Distinct - Qlik Community - 1466796

- Marcus

Re: RAM usage increases when making selection

Chip_Matejowsky — Fri, 13 Aug 2021 14:14:13 GMT

Hi @Messen,

In addition to Marcus's excellent suggestions, have a look at article QIX Engine Memory Management and CPU Utilization. It does a great job in detailing how QlikView utilizes RAM.

Best Regards

Re: RAM usage increases when making selection

Brown3391 — Mon, 16 Aug 2021 04:12:12 GMT

@Messen wrote:
Hi
I have a fairly large QlikView app. Around 85 mill. lines in Facttable. Uses about 15 GB of RAM when opened.

My Straight table has no calculated dimension and 1 expression = '1' for showing the lines.

So no calculations at all. Just showing lines. Very simple.
When making a selection in one field it takes about a minut for showing the data. The RAM usage of the app increases to more than double up - 35 GB. When data showes up in the table - RAM decreases again to 15 GB.
It doesn't matter whether my selection returns 1 or e.g. 50 lines in the table and doesn't matter whether I'm using Desktop or QlikView Server. Version are 12.10. (I know it is outdated....;-)
Tried different Datamodels : Star with 3 dimensions and all in one Fact table. Which surprisingly enough is faster than the model with 3 dimensions.

I have 25 fields. One field is a text field with average lenght of 100 and max lenght of 1000.

Anyone with suggestions why QlikView has to "unpack" so much data in the RAM for making a simpel selection and best of all - Suggestions for speeding up the time for the app.

Thank you in advance for every input 🙂

Regards

Sune

Follow this steps it might help you

Restart Your Computer. The first thing you can try to free up RAM is restarting your computer.
Update Your Software.
Try a Different Myloweslife Browser.
Clear Your Cache.
Remove Browser Extensions.
Track Memory and Clean Up Processes.
Disable Startup Programs You Don't Need.
Stop Running Background Apps.

Re: RAM usage increases when making selection

Messen — Mon, 16 Aug 2021 11:42:31 GMT

Dear All

I’m aware of the caching that makes Qlik a unique product. But the behaviour that I experienced is/was not normal compared to my experience with Qlik.

I found a solution. Not WHY the problem was there.

It was my Straight Table which was ‘fuc…’ 😉

Made a new table with same dimensions and the very simple expression. Deleted old one and speed is now what I expect from Qlik. So - something behind the scenes had gone wrong with the old straight table.

Thank you all for your contribution ! I’m now a lot better informed about the Qlik cache and Qix engine after reading all the articles 😉

Regards

Sune