topic Re: RAM Memory utilisation in QlikView

RAM Memory utilisation

Wed, 11 Dec 2013 03:22:46 GMT

Hi guys,

I'm dealing with huge amount of data in Hadoop DB(Nearly 1 TB/day) how can i handle this in Qlikview. as well as i need utilize the RAM in optimized way as much i have to reduce memory. Can any one please guide me in this issue.

Re: RAM Memory utilisation

ngulliver — Wed, 11 Dec 2013 12:35:01 GMT

Hi,

I would recommend pulling the data into QVDs. Once a QVD is created you can interrogate the QVD as a data layer from another QVW. QVDs compress the data into a manageable form, enabling you to strip out unnecessary data and optimize the data. QVD files are stored in a format that mirrors the compression used in memory.

To create a QVD you need a QVW to store it:

STORE <Your Table Name> INTO ..\Data\<Your File Name>.qvd (QVD);

If you want to know more about QVDs and Optimisation, search the internet for the Quick Intelligence website which has a number of blogs on the subject.

Cheers,

Neil

Re: RAM Memory utilisation

Thu, 12 Dec 2013 08:30:37 GMT

Hi Neil gulliver,

Thanks for your Suggestion

But the problem is, i have 1 TB of (QVD data) but my management is ready to allocate 1 GB RAM memory for our process. even if i use QVD its compressed and when my dashboard open , its occupy ~800GB (more or less) .but the ask us to occupy memory as per filter wise.

we decide for document chaining to split data . but its hard to do in development..

Re: RAM Memory utilisation

qlikpahadi07 — Thu, 12 Dec 2013 09:27:38 GMT

Hi Sundar,

In your case you have to split your application let say Region/Area/Field type. This can be automatically done by Qlikview Publisher.

QV Publisher can reload your application and at the same time it will Split the application let say your Application is big enough to load in Memory so if your QVW contains data for all country (Let say: eg India, Sin, Oman, SA,NA,England ....10 countries ) QV Publisher will split the application in to country wise so there will now 10 Different QVW which will automatically get reduces and will reduce load while open and you can split it up to Granular level (your design).

Also for Second though you can simply do it from development but will be Manual work for you but might be helpful.

Open QVW
Make a Default selection you can create (Let say : Current Year or 2 Year and one region).
your dashboard will get filter as per the selection now
Select File from tools ->Reduce data ->keep possible values
You will promt a Mesg click Okay
and Save as the application rename it and then distribute that QVW.

Note: The time you reload the qvw it will reload all data and you have to do all steps again.

hope this works

Re: RAM Memory utilisation

Sat, 14 Dec 2013 10:00:38 GMT

Hi pahadi,

Thanks for your reply im checking this once it successful will update u . i think it will work fine thanks