RAM optimization & Timeout Management

[#1]

Hi all,

I work on a Information System in which the end user can access to one QlikView document among around 10 reduced document (reduced from one master document) from a web portal.

We still use the release QV10SR3 of QlikView Server.

Our documents are quit big (from 100Mo to 800Mo). when these documents are loaded in RAM on our server (either preloaded through QVS or loaded trough user request), the documents take a lot of space in RAM (for a 800Mo document -> 5,6Go RAM).

Our server has only 32 Go of RAM for the moment. We will upgrade it in the next two months.

I try to find some solutions to optimize the management of the space in RAM.

1/ We try to find the best parametrization of pre-loaded documents according to our users and the consultation needs.

2/ I think our timeout parameters are not optimized:

     - The document timeout on QVS is set to 240 minutes (QEMC: System > Setup > QlikView Server > Documents)

     - The Maximum Inactive session time of the documents are set to 1800 seconds (Document properties > Server)

Why document timeout is so big (240 minutes) I think 60 to 120 minutes could be enough. What do you think ?

What is your best practice about the timeout management ?

Do you imagine other thinks to deal with in order to have the best optimization of the space in RAM ?

Thanks a lot

Benjamin

[advait_thakur]

Hello Benjamin,

  First of all i would like to tell you what Qlikview engine does inside.

QV server gathers/accumulates memory as end users continues to use QV and simultaneously the consumption of memory increases. QV server by default gives between 75 to 90% of the total RAM (32Go in your case).

For particular users query will be requested as they executes. When QV server touched the limit of 90% of total RAM, it will begin to come to normal and recycle it.

I would recommend you to increase the RAM as early as possible.

Yes normally, 60 to 120 minutes are sufficient.

Thanks,

Advait

[#3]

Hi Advait,

Thanks for your answer.

Could you tell me more about the RAM memory settings (75 to 90%) ? Why is there a low limit ?

What I see on my server is:

- when a document is loaded in memory, the memory increases to 6-7 times the size of the qvw file (about 5-6Go).

- when a user is working on the document, the memory goes up and down:

     - 120Mo when the user opens the document and a max to around 1,2Go.

     - When the session is closed, the memory decreases after the Maximum Inactive session time of the document (but the memory RAM does not go back to initial state, around 100Mo are remaining)

     - When the session is closed, the memory decreases after the document timeout of QVS, and loose about the 5-6Go corresponding to the document load in memory.

The recycling you are describing, is when I imagine you have a lot of different users simultaneously, the part of memory allocated to user consultation can be big and QV engine has to manage priority among the different current and "stored in memory" requests of users. But how does QVS manage a load of documents which  Am i true ?

In my case, the problem i am facing is more about the large memory space allocated to opened documents.

I have not a lot of simultaneous consultation. For example I can have 5-6 different people which consults different documents in the same days. The average session duration is about 1h30.

Because of the large size of my documents, the RAM grows to 25Go of RAM frequently only with the space memory linked to the document load.

Currently, the timeout document set on my production server is 480 minutes and the performance decreases during the day.

Thanks for new advice

[advait_thakur]

Hi Benjamin,

  What you mentioned few 2-3 points is true.(Indicating "-")

Your windows is responsible to allocate RAM for Qlikview. QV server always attempt to reserve RAM when it gets started based on the 'Working Set Limits' in Qlikview Server Management Console.
If RAM's lack while opening document or reloading then it swaps and uses Virtual memory.

I guess its taking large time to open the documents, may be one of the reasons below.

Actually it is very important to understand that the data which is stored in your RAM is the unaggregated data or not. I hope in your case no preaggregation is performed while execution or reloading process.

I would like to know about the size of your document ..!!
Size matters a lot, the size even vary's what joins you have consider while Scripting.

Also will you please try to partial load and open the document once. (Observe how much time it takes)

Thanks,

Advait