15 Replies Latest reply: Apr 20, 2012 5:00 AM by Hugonas Petkus RSS

    performance

      Hello

      By idea of Rob I make very simple qvw document:

      Load Recno( ) as A, Rand() as B autogenerate(10000000);

      Reload on my local notebook is about 25 sec long.

      Reload on our huge servers is about 46 sec long ! (servers are iddling before starting reload).

      Offer:

      Can other users respond how long is reloading this simple qvw on yours servers ?

       

      Thanks

       

        • performance
          Karl Pover

          Hugo,

          19 sec in a notebook with Windows 7, 8 GB RAM and 8 CPU system threads, but my notebook is also faster than some of the bigger servers. See the my following post.

          http://community.qlik.com/forums/t/33509.aspx

          Regards.

            • performance

              Hello

              Using 10 version reload duration on local PC is 15 sec !

              good reason to upgrade to 10 version.

              best regards

              Hugo

                • performance

                  Just tried this test on my machine and it takes 9 seconds :P

                  Whilst I am using QV10 and using all cores (i7 920 cpu which has a clock speed of 2.67GHz), I would also mention that the bus speed between CPU and RAM is 1GHz so it should be fast....... not to mention having overclocked RAM :-)

                  Some of these servers are quite big and have lots of resources, but they are usually tailored to throughput and capacity rather than sheer response speed. A reload that is CPU heavy and is trying very hard to load things into RAM might take longer on a server that has a lot of cores but with lower clock speeds/specifications. However when scaling and serving hundreds of users it will do a better job than a small desktop with a single core running at 3.2GHz. Horses for courses.

                • performance

                   

                  Laptops are optimized for low power usage = slow disks, narrow buses, slow memory, small CPU cache.

                  If the "idle big servers" are slower than a notebook, throw them away.

                  -Alex



                    • performance

                      I agree with Alex, makes no sense having better response times on laptop. We are experiencing this kind of behaviour in a customer right now, and I can tell you the customer is pissed off. We ran testing software on the server to discard any failure on hardware, the test did not show any failure, on the other hand I showed that the server was performing better on cpu intensive calculations and access to RAM, since the processor has a 4-memory-channel. I'm suspecting that the current implementation of QlikView has problems dealing with several cores and multi-threading, my theory is that probably the software is dividing the task into so many threads that the amount of actual work for every thread is little and the overhead of communication probably is over killing performance. On the other hand, it might not be QlikView itself, but the Windows scheduler that might not be ready to successfully allocate this type of tasks.

                      Regards

                        • performance

                          Hi,

                          Disable CPU hyperthreading. Suppose there is sizeable amount of memory, so don't let Windows automatically manage the size of the pagefile. Use a fixed amount of pagefile, ex 2 GB. The old mantra swap = two times the memory does not apply for a tool called "in memory analytics"; you really need data to fit in memory.

                          To do a one-to-one comparison with laptop, limit QVS service from using all CPU cores from QEMC : System : Setup: QlikView Servers : Performance : CPU affinity. Choose same amount of cores as in the laptop. Use the Windows task manager to really see the rest of the CPU cores idle.

                          -Alex

                            • performance

                              Hi Alex, we have tried disabling hyper threading and reducing the CPU affinity, the results we got were better but not really what we expected. We have run test in 5 different servers, so far we have had the best results on xeon quad core processors. The next step is try in another server with several quad core processors and see if the performance improve comparing the single quad core cpu architectures we had tried. I suppose having cpu with more than four cores mess up performance because of the shared cache memory, imagine every core trying to write on the shared buffer at once. Anyway, we will let you know how this issue ends.

                              Regards.

                                • performance

                                  Instead of setting affinity 11110000 00000000, set 10001000 10001000 . That uses only real CPU cores, not threads

                                  -Alex

                                    • performance

                                      Thanks Alexandru, for some reason QlikView will only allow to reduce the number of cores by half maximum. I'll continue do testing.

                                      Regards

                                        • performance
                                          Nicolas MARTIN

                                          Same thing here :

                                          - My laptop is Core 2 Duo @ 2.33 GHz with 2 Gb of RAM (Windows XP) : 24 seconds to reload (QV 9 SR 6).

                                          - My client server has 16 cores with 32 Gb of RAM (Windows 2008 Server SP 2) : 36 seconds to reload (QVS 9 SR 6).

                                           

                                          I tried to disable hyperthreading, that makes the process slightly slower.

                                          Tongue Tied

                                            • performance

                                              To make a fair comparison, set your idle server to use only 2 out of 16 cores with affinity mask = 00000001 00000001 (binary) = 0101 (hexa)

                                              Schedule a batch file OR type every now an than during the testing in PowerShell

                                               


                                              powershell -command "& { get-process qvb | foreach{ $_.processoraffinity=0x0101} }"


                                              -Alex