topic Re: Out of Memory in a straight table in QlikView

Out of Memory in a straight table

Thu, 19 Sep 2013 11:45:41 GMT

Hi all,

I am pretty new to QV but this community has been very helpful. I have two questions I need answers to.

My dataset is pretty huge (18 lakh rows). And, in a straight table I am displaying output based on selections made. This table will have around 15 columns max. I made sure that some selection is made on the sheet so that this object doesn't run out of memory. However, when multiple selections are made (say, Countries 1 2 and 3 are selected together), this table runs out of memory.

Q1: Is there a way to avoid this?

Q2: We are planning to host this using QV publisher license. Since it is on a server there will be higher RAM alloted. So will this object be able to run on the server without losing memory?

Kind of urgent. Please help.

Prem

Re: Out of Memory in a straight table

jagan — Thu, 19 Sep 2013 11:52:37 GMT

Hi Prem,

Try to restrict the user to select as much of as selections, may be you are working in your machine with low configuration, if you deploy this in server with huge configuration there you will not get this error.

If the same exists in server also then use

Calculation Condition in General Tab of Chart properties, if you have any ID field then use like this

=Count(ID) < 10000

This will restrict the user to limit the valid records to display in chart by 10000 records, if it displays faster then increase the count.

Hope this helps you.

Regards,

Jagan.

Re: Out of Memory in a straight table

Thu, 19 Sep 2013 12:51:25 GMT

Exactly as Jagan said. Restrict the table and please notify the end users to apply more filters. Do this in table Properties>General>Error Messages>Calculation condition unfulfilled>Custom Message: ='Please apply more filters. Select less then 10 000 rows. Now are selected: '& num(sum(Row),'# ##0')&' rows.'

Another solution would be move your calculation from Expression directly to loading script.

Dusan

Re: Out of Memory in a straight table

Thu, 19 Sep 2013 15:45:12 GMT

THANKS!

Re: Out of Memory in a straight table

Thu, 19 Sep 2013 15:45:30 GMT

Thanks Jagan

Re: Out of Memory in a straight table

Sat, 28 Sep 2013 17:27:47 GMT

Well, It is strange even when there are no calculations in straight box it consumes 10 times more memory than table box

Re: Out of Memory in a straight table

rajeshvaswani77 — Sun, 29 Sep 2013 04:40:13 GMT

Hi,

If it serves your purpose, you can use a table box. That's the lightest.

thanks,

Rajesh Vaswani

Re: Out of Memory in a straight table

richard_chilvers — Mon, 30 Sep 2013 08:41:17 GMT

Memory issues can arise for a number of reasons, including the hardware being used. In my experience, memory problems have usually arisen when the data structure from the script is not quite as anticipated.

It may be worthwhile checking to see whether your script is working excatly as you expect.

Regarsd

Re: Out of Memory in a straight table

Tue, 04 Feb 2014 07:53:35 GMT

Hi,

The best way to resolve this issue. You need to tune your expressions accordingly ans also the script.

Ben

Re: Out of Memory in a straight table

Mon, 04 Aug 2014 00:45:58 GMT

Hi Jagan,

I have calculation condition unfulfilled error with this expression : =Count(PostID) < 100.

By the way, inyour expression, if the ID is more than 10000, then it will only consider only the first 10000?

For example, count(ID) = 25000

1. without filter = the straight table will show first 10000 ID

2. With a filter (decrease count(ID) to 15000 = the straight table will still show first 10000 ID

3. With 2nd filter (decrease count(ID) to 9000 = the straight table will show all 9000 ID.

How if i want to show first 1000 with descending date in calculation condition?

I have the same problem in my big data application, the row is in million count, so i face the same issue.

Thanks,

Chanel

Re: Out of Memory in a straight table

jagan — Mon, 04 Aug 2014 02:13:14 GMT

Hi Chanel,

=Count(PostID) < 100

By using the above condition we can restrict the table to not show more than 100 rows, for example for your selections if you have more than 99 rows then you will get calculation condition unfulfilled error message, otherwise it will show the data in the table.

By the way, inyour expression, if the ID is more than 10000, then it will only consider only the first 10000? -- It won't consider the first 10000, the count depends on the selections made in your dashboard.

How if i want to show first 1000 with descending date in calculation condition? -- Sort the table by date in descending order and in Dimension limits give 1000

Regards,

Jagan.

Re: Out of Memory in a straight table

Mon, 04 Aug 2014 07:00:32 GMT

As shared in other comments, you can limit the data using calculated condition. But there are many other reasons for your straight table to go out of memory.

1) Your expressions must be optimized. Use set analysis.

2) Avoid using calculated dimensions.

3) Avoid using Primary keys as dimensions.(i read somewhere)

4) The data model must be efficient for the requirements.

Thanks,

Angad

Re: Out of Memory in a straight table

Mon, 04 Aug 2014 07:06:09 GMT

third point is really funny where did you read that

Re: Out of Memory in a straight table

Mon, 04 Aug 2014 07:15:13 GMT

Hi Angad,

Thanks for your reply! My document consists social data of million rows.

1. Only one expression with set analysis

= if( ShortCreatedDate >= $(vStartDate) AND ShortCreatedDate <= $(vEndDate),BuzzVolume)

2. No calculated dimension, unless

=date(CreatedDateTime, 'dd-MMM-yy')

is considered as one...

3. I am not sure about that as i use synthetic keys in dimension

4. data model is considered simple.

Thanks,

Chanel

Re: Out of Memory in a straight table

jagan — Mon, 04 Aug 2014 07:25:07 GMT

Hi Chanel,

1. Only one expression with set analysis

= if( ShortCreatedDate >= $(vStartDate) AND ShortCreatedDate <= $(vEndDate),BuzzVolume)

replace with

Only({<ShortCreatedDate ={'>=$(=vStartDate)<=$(=vEndDate)'}>}BuzzVolume)

2. No calculated dimension, unless

=date(CreatedDateTime, 'dd-MMM-yy')

Format the date in script so that you can avoid calculated dimension.

Hope this helps you.

Regards,

Jagan.

Re: Out of Memory in a straight table

Mon, 04 Aug 2014 07:41:34 GMT

Awesome! hope it can work well!