RAM Memory utilisation

[#1]

Hi guys,

I'm dealing with huge amount of data in Hadoop DB(Nearly 1 TB/day) how can i handle this in Qlikview. as well as i need utilize the RAM in optimized way as much i have to reduce memory. Can any one please guide me in this issue.

[ngulliver]

Hi,

I would recommend pulling the data into QVDs. Once a QVD is created you can interrogate the QVD as a data layer from another QVW. QVDs compress the data into a manageable form, enabling you to strip out unnecessary data and optimize the data. QVD files are stored in a format that mirrors the compression used in memory.

To create a QVD you need a QVW to store it:

STORE <Your Table Name> INTO ..\Data\<Your File Name>.qvd (QVD);

If you want to know more about QVDs and Optimisation, search the internet for the Quick Intelligence website which has a number of blogs on the subject.

Cheers,

Neil

[#3]

Hi Neil gulliver,

Thanks for your Suggestion

But the problem is, i have 1 TB of (QVD data) but my management is ready to allocate 1 GB RAM memory for our process. even if i use QVD its compressed and when my dashboard open , its occupy ~800GB (more or less) .but the ask us to occupy memory as per filter wise.

we decide for document chaining to  split data . but its hard to do in development..

[qlikpahadi07]

Hi Sundar,

In your case you have to split your application let say Region/Area/Field type. This can be automatically done by Qlikview Publisher.

QV Publisher can reload your application and at the same time it will Split the application let say your Application is big enough to load in Memory so if your QVW contains data for all country (Let say: eg India, Sin, Oman, SA,NA,England ....10 countries ) QV Publisher will split the application in to country wise so there will now 10 Different QVW which will automatically get reduces and will reduce load while open and you can split it up to  Granular level (your design).

Also for Second though you can simply do it from development but will be Manual work for you but might be helpful.

1. Open QVW
2. Make a Default selection you can create (Let say : Current Year or 2 Year and one region).
3. your dashboard will get filter as per the selection now
4. Select File  from tools ->Reduce data ->keep possible values
5. You will promt a Mesg click Okay
6. and Save as the application rename it and then distribute that QVW.

Note: The time you reload the qvw it will reload all data and you have to do all steps again.

hope this works

[#5]

Hi pahadi,

Thanks for your reply im checking this once it successful will update u . i think it will work  fine thanks