9 Replies Latest reply: May 11, 2009 9:42 PM by Rob Wunderlich RSS

    Out Of Memory

    David Inman

      Hi As the subject states I have a table which regularly says Out Of Memory.

      I am wandering how I would solve this problem, in user properties the checkbox to allocate memory automatically is ticked which is the recommended approach however I still receive this message.

      Would upgrading the memory in the server solve this problem?

      Thanks in advance

       

        • Out Of Memory
          Sander Polman

          Hi,

          Upgrading your memory could help. But only if you are running QV on a 64bit platform. On a 32bit platform the Windows OS can't allocate more than 3GB of memory for 1 process. So you could have 32GB of memory but 29GB will not be used by QlikView. There are some hacks for Windows server to allocate more memory but hacks are hacks....

          Another way to avoid running out of object memory is to reduce the number of fields and do not use long text fields, or find a way to reduce the quantity of data by editting your script somehow.

          The Out of memory notification isn't that bad, you could replace it with a custom error message telling the user to reduce his selection or something.. This is done in Chart Properties -> General -> Error messages.

          Regards, Sander

          • Out Of Memory
            Oleg Troyansky

            In most cases, the problem is hidden somewhere in the design of the table:

            1. It might be way too detailed - how many cells are you expecting to populate? If you have a detailed table with half a million rows and 20 columns, no server memory will be sufficient...

            2. You might have fields in the table that are not properly linked, and QlikView is generating a "Cartesian join", linking "everything to everything". This problem obviously needs to be fixed in the data structure.

            3. If it's a Straight Table or a Pivot Table, you might have complex expressions with IF statements and other heavy functions (read my WIKI page about avoiding IF statements).

            4. If it's a Table Box... Well, Table Box is not so good when it comes to performance. I'd recommend replacing it with the Straight Table, but still look for one of the problems listed above.

            If you want to reduce data and post a sample, we could take a look...

            Oleg

              • Out Of Memory
                Michael Solomovich

                To add to Oleg's #1:
                Ask the clients (users), do they really want to see that million, or even 10,000 rows? Usually the answer is no. So, you can safely use some calculation conditions and don't bother to make calculations that return too many rows. Don't forget to replace the default error message "calculted condition unfulfilled" with something more meaningful, like "...make selections...".

                Oleg, 20 colums, you say. Smile... How about a straight table with several hundred columnsConfused

                  • Out Of Memory
                    Oleg Troyansky

                    MIchael,

                    God help you and your users :-)

                    Oleg

                      • Out Of Memory
                        Michael Solomovich

                        Oleg, believe it or not, they are happy with it...
                        (IMHO it's rather reporting with elements of data mining, but not BI.)

                          • Out Of Memory
                            Oleg Troyansky

                            it's not even that - it's a fancy interface into Excel. I bet your users are using QlikView as their ETL process, and then they dump your hundreds of columns into Excel, and then the real analysis begins... Very sad... Either a training issue, or an attempt to save on licenses...

                              • Out Of Memory
                                David Inman

                                Hi Thanks for all of your replies.

                                We are running a 64 Bit platform with 16 GIG of RAM if we upgraded the RAM would that solve the problem?

                                I know you recommend to only allow the users to view a subset of data, and your correct the user will not want to view all records but for the purpose of the application created we would prefer to solve the 'out of memory problem' whereby the pivot will be viewable until the user decides to filter.

                                 

                                Thanks

                                 

                                  • Out Of Memory
                                    Sander Polman

                                    Well.. Adding another 16GB of memory would help.... 16GB of memory to be exact.. Big Smile But as Oleg says, this still strongly depends on the volume of data you wish to display..

                                    You say you are using a pivot. How many levels of pivotting does the table contains? And what is the bottom level, transaction level or still a cummulation?

                                    A solution could be to add a conditional show to the pivot. Where it doesn't show before the user makes a couple of selections. Filtering first should be a logical step in this case offcourse..

                                    Regards, Sander

                                    • Out Of Memory
                                      Rob Wunderlich

                                       


                                      onlyone wrote:
                                      We are running a 64 Bit platform with 16 GIG of RAM if we upgraded the RAM would that solve the problem?
                                      I know you recommend to only allow the users to view a subset of data, and your correct the user will not want to view all records but for the purpose of the application created we would prefer to solve the 'out of memory problem' whereby the pivot will be viewable until the user decides to filter


                                      I think before you upgrade RAM, you'll want to determine if the current memory problem is best case or if you have an application design problem. 16GB is a pretty good size machine and should be able to handle a fairly large application.

                                      How many rows are in your data tables?

                                      Can you upload a reduce sample of the app so we can take a look and offer some suggestions for tuning?

                                      -Rob