topic Out of Object Memory in QlikView

Out of Object Memory

Fri, 22 May 2009 21:19:57 GMT

I am using 8.5 and I have a qvw file which I consider to be quite small at 7MB in size. All the objects run satisfactorily in summary mode but when I try to create a transaction level object I always get "Out of object memory". There seems to be a limit of less than 2500 records!!!!

I have tried the following:

Using getPossibleCount to restrict the data works OK but I would like to remove this restriction.
Settings > User Preferences > Objects tab > Calculated Formulas. Incremented from 1MB to 10MB.
Settings > User Preferences > Objects tab > Sort By Express. Incremented from 2MB to 10MB
Settings > User Preferences > Export tab > Export Memory. Increased from 1024MB to 1500MB

I do not understand what the problem is and how I can diagnose how to fix it.

Any assistance would be appreciated.

Out of Object Memory

sathishkumar_go — Fri, 22 May 2009 22:19:07 GMT

Plz check any syncronous key is there in ur application?

I am not sure may be some time?

Out of Object Memory

Fri, 22 May 2009 22:24:55 GMT

Unfortunately there are no sync keys. Everything looks OK. Thanks for the thought

Out of Object Memory

Anonymous — Fri, 22 May 2009 23:00:46 GMT

Hi Peter,

It's hard to say what the issue is right off the bat like this. The actual file size usually has little to do with the amount of RAM consumed when calculating objects. I would suggest sending this document to support.

However, here's a few things that are usually present in objects where Out of object memory occurs.

Several dimensions & Expressions, Calculated dimensions, "Expensive" expressions (Count distinct, string comparison, nested conditions based on calculations), Pivoted pivot tables, Something strange with the data structure. Unfortunately it's very hard to say what might be causing the issue in this case but I've yet to seen a document where changing the internal memory settings would be required for an object to calculate.

Out of Object Memory

sathishkumar_go — Fri, 22 May 2009 23:22:05 GMT

Hi Pieter

One more suggestion go for ur system properties --> Advanced --> Virtual Memory

In that defauflt 2mb is allocated for all system.

u change to maximum then try

Regards

Sathish

Out of Object Memory

Tue, 26 May 2009 15:19:47 GMT

Thanks for the information. As I cannot see it is any of your suggestions, which I have noted for future, I will send my file to support

Thanks you for taking your time to respond

Out of Object Memory

Tue, 26 May 2009 15:22:16 GMT

Hi Satish

Thanks for the information. I have now tried this and unfortuantely this did not work either.

Thanks you for taking the time to respond I am going to send the file to support. I will inform you all of the answer I get..

Peter

Out of Object Memory

Fri, 09 Oct 2009 17:46:05 GMT

@ Peter Crossly

Did u get any feedback. I am facing the same problem too.

Regards

Sravan

Re: Out of Object Memory

Thu, 05 Oct 2023 12:36:28 GMT

Normal 0

Out of Object Memory
This error message can occur for several reasons. Is this happening during a script reload or when a chart or object on the page is trying to reload? If it's a chart it is because of your expressions. You either have to much data, to long, or a really bad if statement in one of your expressions. Check that out, and let me know. If it's a chart please post your expressions so I can have a look at it.

Thanks,

Anthony

Out of Object Memory

Thu, 15 Oct 2009 17:35:01 GMT

Hi Sravan

I received the following response from UK Support:

I would suggest as a first step, you look at the number of fields being included in the application and validate that they are all required. Fields loaded but not referenced in any sheet objects will still use memory. I would then suggest looking at the data structure as there are quite a few tables which could require QV to link across each of these tables to locate all the items in your display table. I think it is the structure of the data model rather than the volume of records which is causing your problem."

The information sent to me below provides some useful techniques for improving application performance.

Hope this helps you as it helped me

Best regards

Peter

Author: ABY, QlikTech UK.

Performance Checklist…5 quick wins…

A QlikView application can be constructed in different ways and still present the user with the same results.

However, there are a few pointers below to aid in speeding up your analysis……

1 - Memory Intensive Expressions

Wherever you see the ellipsis within QlikView you can ad a dynamic/database object/function. This does also mean that you can add complex formulas and functions to most parts of your application, including Titles, text objects, chart headers etc etc. You must remember that any object or item that requires a calculation, takes up some memory.

2 - Pivot vs Straight Table

Pivot tables give the user a massive amount of flexibility but basic straight tables are less memory intensive in large data volume implementations.

3 - Many leaf data structures

If, within your data model, you have a large amount of tables that simple contain 2 columns (usually used for codes and descriptions), you could look to using the Mapping Load and Applymap functions to pull the leaf tables closer to the fact table therefore reducing the model complexity.

4 - Remove unused data from the model

Removing data that is not required from your data model can reduce the volume and uniqueness of the data model. This reduced uniqueness and smaller model will mean the application has a reduced footprint in memory and result in faster analysis. You can simply comment the columns out to ensure that if they are needed at a later date you can easily readd them.

5 - Script side IF Statements

In most applications it is absolutely fine to add IF statements and other formulas to your charts, but in larger data volume apps or in areas where memory power is restricted you may be able to add the data to the script…for example, in a chart you may wish to just view sales for Product ABC so the expression will be something like…..

SUM(IF Product = 'ABC',Sales,0)

You could move this line to the script and create an ABC Sales column….

IF(Product = 'ABC',Sales,0) as ABC_Sales

The chart can then simply use

SUM(ABC_Sales)

as the expression.

Out of Object Memory

Thu, 15 Oct 2009 17:37:05 GMT

Hi Anthony

I changed my load scriptis and used information provided from UK Support. Pleassee my reply I have just posted to Sravan

Best regards

Peter

Out of Object Memory

Thu, 03 Dec 2009 00:19:03 GMT

I have the same problem - it happens during a chart trying to reload - the data is big - around 400 millions

the Expression of the chart: if (Age='P 18-49',sum(P_18_49_Cost) ,sum(P_25_54_Cost))

And Age is the island I created - so user can select whether to choose P 18-49 or P 25-54

Out of Object Memory

johnw — Thu, 03 Dec 2009 03:59:26 GMT


Senjaya wrote:if (Age='P 18-49',sum(P_18_49_Cost) ,sum(P_25_54_Cost))

In your case at least, it almost certainly isn't the IF. Since it is outside of your aggregation, the IF should only be checked once, not once per record.

Out of Object Memory

Thu, 03 Dec 2009 16:09:12 GMT

Hi Senjaya.

I would see if the expression: sum(if (Age='P 18-49', P_18_49_Cost ,P_25_54_Cost) works any better

Peter

Out of Object Memory

Thu, 03 Dec 2009 16:11:30 GMT

Hi Senjaya

Another thing to consider is a conditional load of the chart, as it will evaluate all 400 milion transactions all of the time.

Peter

Out of Object Memory

Thu, 03 Dec 2009 21:12:32 GMT

There were 2 table in the database one is local and 1 is national - but in Qlikview we combine them into 1 fact table - that's why it reach the 400 millions records - but the client can only can choose 1 and not both. - Is there any good way to separate both table into 2 separate fact table ?

Out of Object Memory

Thu, 03 Dec 2009 22:38:02 GMT

Hi Peter,

Thank you very much for the help - the sum(if) statement definitely was the fixed. - It removed the problem - now the chart work fine.

Regards,

Senjaya