topic Re: How to drop document from memory manually? in QlikView

How to drop document from memory manually?

Wed, 09 Jul 2014 18:20:18 GMT

Hi All,

Could anyone suggest how to drop qlikview document from server memory using batch script? I have the requirement to drop existing document from server memory manually before publishing the refreshed document.

Re: How to drop document from memory manually?

whiteline — Wed, 09 Jul 2014 18:27:08 GMT

Hi.

What is the reason for that ?

The server should do it automatically.

Re: How to drop document from memory manually?

hic — Wed, 09 Jul 2014 18:45:06 GMT

I agree with whiteline. If you check "Allow Only One Copy of Document in Memory" in the QMC, then the server will handle this automatically.

HIC

Re: How to drop document from memory manually?

mikkeltaylor — Wed, 09 Jul 2014 18:50:45 GMT

If the data is refreshed from the database why would you want to keep that data in memory? What would qlikview do with outdated data?

Re: How to drop document from memory manually?

Wed, 09 Jul 2014 19:02:53 GMT

My server is crashing because of insufficient memory when reloading/publishing refreshed document. If I drop document from memory before publish the new refreshed version that will free up lot of memory so that I won’t crash the server due to lack of memory.

Let’s assume my server has 100 GB RAM, and my qvw file size is 10GB compressed. Before reloading document assume I have only 20GB available in server as other documents are already preloaded in memory. So there is no sufficient memory to reload and publish this updated document.

Re: How to drop document from memory manually?

Wed, 09 Jul 2014 19:07:48 GMT

That option is already checked for my server. Unless publisher process is at final step it does not drop the existing one/old snapshot. But i am getting server crash while refreshing/reloading document itself due to insufficiant memory. That is the reason , I want to drop the preloaded document before I start reload.

Re: How to drop document from memory manually?

whiteline — Wed, 09 Jul 2014 19:29:03 GMT

Ok.

How did you noticed that the insufficient memory caused the problem ?

You can read the document about how the QV cache works. The most reason for the server to keep the cache and the old document in memory is that someone is using it.

You can try to cheat and temporarily replace the document with the one that has empty data set.

Then you can start the real reload process. Of course it can lead to some inconsistencies.

Usually the server flush the unnecessary cache and frees the memory when some other process needs more memory.

Re: How to drop document from memory manually?

whiteline — Wed, 09 Jul 2014 19:32:17 GMT

You can search for the document that Henric Cronström has published about how the QV cache works.

Re: How to drop document from memory manually?

Thu, 10 Jul 2014 13:47:51 GMT

My server team notifying that server was keep on hanging due to over usage of CPU and Memory by qlikview software. My refresh of tasks runs midnight of the day so no user is going tobe online during that time.Also i just want to drop document from memory, not from accesspoint portal folder. If incase anybody logs into server at midnight still it should be able to load that document into memory anytime. But nobody logs into the server that time.

Thank you. That is very good article from Henric. Still that doesnot resolve my issue

Re: How to drop document from memory manually?

sudeepkm — Thu, 10 Jul 2014 13:55:46 GMT

Have you tried keeping a less duration (like 5 min) Document timeout.

In that case if QlikView Server will drop the document from memory when no one is using it for more than 5 min.

Re: How to drop document from memory manually?

Thu, 10 Jul 2014 14:24:12 GMT

If i change that settting, that will apply to all the documents. But I want to drop only couple of big qvw files before reloading with new data only.Rest of the time I dont see any problem having them in memory.That is the reason I am looking for some script, where we can specify which document to drop(like batch scrip). Sothat I can schedule that script as external task job before refreshing the qvw document.

Re: How to drop document from memory manually?

rwunderlich — Thu, 10 Jul 2014 14:45:36 GMT

My first recommendation would be to move publisher to a separate server. Your server is over committed.

If you can't separate the two, consider lowering the working set limits in the QMC to keep more memory free.

After that, i think Whiteline's idea of replacing the doc with a small dummy could be useful.

-Rob

Re: How to drop document from memory manually?

whiteline — Thu, 10 Jul 2014 21:38:43 GMT

Hi.

Have you tried to do some tests and profiling? Do you know exactly what operation during reload process consumes so much memory? And do you have the reason to do it in that way?

Re: How to drop document from memory manually?

Fri, 11 Jul 2014 14:32:03 GMT

I got the temporary solution to not to crash the server. I am preloading these large qvw files in specif cluster node( where there is no QDS and QMS installed) instead of all nodes. That way reload server(Where QDS and QMS installed) always have some free space to refresh the document.