Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi
I have a fairly large QlikView app. Around 85 mill. lines in Facttable. Uses about 15 GB of RAM when opened.
My Straight table has no calculated dimension and 1 expression = '1' for showing the lines.
So no calculations at all. Just showing lines. Very simple.
When making a selection in one field it takes about a minut for showing the data. The RAM usage of the app increases to more than double up - 35 GB. When data showes up in the table - RAM decreases again to 15 GB.
It doesn't matter whether my selection returns 1 or e.g. 50 lines in the table and doesn't matter whether I'm using Desktop or QlikView Server. Version are 12.10. (I know it is outdated....;-)
Tried different Datamodels : Star with 3 dimensions and all in one Fact table. Which surprisingly enough is faster than the model with 3 dimensions.
I have 25 fields. One field is a text field with average lenght of 100 and max lenght of 1000.
Anyone with suggestions why QlikView has to "unpack" so much data in the RAM for making a simpel selection and best of all - Suggestions for speeding up the time for the app.
Thank you in advance for every input 🙂
Regards
Sune
Dear All
I’m aware of the caching that makes Qlik a unique product. But the behaviour that I experienced is/was not normal compared to my experience with Qlik.
I found a solution. Not WHY the problem was there.
It was my Straight Table which was ‘fuc…’ 😉
Made a new table with same dimensions and the very simple expression. Deleted old one and speed is now what I expect from Qlik. So - something behind the scenes had gone wrong with the old straight table.
Thank you all for your contribution ! I’m now a lot better informed about the Qlik cache and Qix engine after reading all the articles 😉
Regards
Sune
The behaviour you noticed is called caching and is one of the main-features of Qlik - means most of calculation-results from selections are kept within the RAM and if this selection comes again it won't be new calculated else the cached results will be used. In many scenarios it's a real benefit - but it depends on the underlying data and the kind of analysis.
In your case Qlik tries to cache but discard it immediately because it hits any restriction which I don't know in detail. More background could you find here: The QlikView Cache - Qlik Community - 1468270
Beside this it's useful to limit the rows of large tables with a visibility-condition like:
count(distinct Dim1&Dim2&Dim3) < 10000
Also helpful should be to switch from the straight-table to a table-box which is AFAIK significantly faster as the table-charts.
Further I suggest to look on the cardinality of your fields especially for your mentioned text-field. The RAM consumption is directly related to the max. number of distinct field-values and you may be able to split the field and/or remove not really important information. The logic behind it is here very well explained:
The Importance Of Being Distinct - Qlik Community - 1466796
- Marcus
Hi @Messen,
In addition to Marcus's excellent suggestions, have a look at article QIX Engine Memory Management and CPU Utilization. It does a great job in detailing how QlikView utilizes RAM.
Best Regards
@Messen wrote:Hi
I have a fairly large QlikView app. Around 85 mill. lines in Facttable. Uses about 15 GB of RAM when opened.
My Straight table has no calculated dimension and 1 expression = '1' for showing the lines.
So no calculations at all. Just showing lines. Very simple.When making a selection in one field it takes about a minut for showing the data. The RAM usage of the app increases to more than double up - 35 GB. When data showes up in the table - RAM decreases again to 15 GB.
It doesn't matter whether my selection returns 1 or e.g. 50 lines in the table and doesn't matter whether I'm using Desktop or QlikView Server. Version are 12.10. (I know it is outdated....;-)
Tried different Datamodels : Star with 3 dimensions and all in one Fact table. Which surprisingly enough is faster than the model with 3 dimensions.
I have 25 fields. One field is a text field with average lenght of 100 and max lenght of 1000.
Anyone with suggestions why QlikView has to "unpack" so much data in the RAM for making a simpel selection and best of all - Suggestions for speeding up the time for the app.
Thank you in advance for every input 🙂
Regards
Sune
Follow this steps it might help you
Dear All
I’m aware of the caching that makes Qlik a unique product. But the behaviour that I experienced is/was not normal compared to my experience with Qlik.
I found a solution. Not WHY the problem was there.
It was my Straight Table which was ‘fuc…’ 😉
Made a new table with same dimensions and the very simple expression. Deleted old one and speed is now what I expect from Qlik. So - something behind the scenes had gone wrong with the old straight table.
Thank you all for your contribution ! I’m now a lot better informed about the Qlik cache and Qix engine after reading all the articles 😉
Regards
Sune