31 Replies Latest reply: Sep 27, 2013 9:08 AM by Hampus von Post RSS

Server with 2 sets of CPU's

Karl Pover

Hello,

 

We're reviewing a server with 2 sets of CPU's.  Each set is on a different card and has 4 CPU's.  Anyone ever have problems with this hardware configuration?

 

Thanks and regards.

  • Server with 2 sets of CPU's
    Stefan Bäckstrand

    Can you elaborate a bit more? What do you mean with sets? Cards? Are there any tech specs?

     

    I haven't seen such convoys myself, if I don't misunderstand the description. But that's only me. :)

    • Re: Server with 2 sets of CPU's
      Karl Pover

      Hello Stefan,

       

      The machine is a HP ProLiant DL980 G7 with 8 8-core Intel X7560 processors on 2 separate trays and 1 TB of RAM.  I've attached more detailed specifications.  We were seeing poor performance a couple weeks ago, but HP seems to have fixed it because the performance has improved.

       

      Regards.

      • Server with 2 sets of CPU's
        alex.archebit

        cpu hyper-threading is enabled ?

      • Server with 2 sets of CPU's
        Pablo Labbe

        Hi Karl,

         

           Did you solved the performance problem ?

         

        Regards,

         

        Pablo

        • Server with 2 sets of CPU's
          Karl Pover

          Pablo,

           

          Sadly, the performance improved without knowing what the hardware provider did.  We improved by a couple more seconds using info from tip #8 http://community.qlik.com/docs/DOC-2362

           

          My advice to anybody who believes they have a server performance issue is to look at the QV app itself.  I've seen response times reduced 50-60% by changing the RAM configuration or BIOS, but that never seems to be enough for user that expects 1-2 second response times.  To achieve those type of response times, you probably need to reduce the data volume or the complexity of the expressions.

           

          Karl

          • Server with 2 sets of CPU's
            Pablo Labbe

            Hi Karl !

             

              The referenced document are part of a valuable section of the Qlikview Scalabity Group that I´ve seen it before. Thank you.

             

              However, can you share with us how large are your environment that requires this kind of hardware ? A server with 1 TB of RAM isn't a common one.

             

            Best Regards,

             

            Pablo Labbe

            Qlikview Consultant

            www.visiongi.com.br

            • Server with 2 sets of CPU's
              Karl Pover

              Hi Pablo,

               

              It's not so much that the environment is large.  I've only ever seen 120 GB being used at one time.  I haven't analyzed the log data, but there are 138 Named CAL's and 600 Document CAL's.  The source data has billions of rows of data, but we're using QV apps with weekly aggregated data reduced to the last 2 years in most cases. 

               

              The server is so large because of the potential number of users and data in the next couple of years.  Looking back on the whole evolution of QV, in my opinion, it would have been better to purchase a server half that size and add an additional clustered in the future.

               

              Karl

              • Server with 2 sets of CPU's
                Ashutosh Paliwal

                Hi Karl,

                Yes clustering two servers with 512gb of RAM would have been better choice. Because, some days back we just had a discussion with someone from Qliktech and he suggested that performance does not go well with more than 512gb of RAM, so in these cases he suggested clustered enviornment.

                 

                Just my 2 cents.

                 

                ..

                Ashutosh

              • Re: Server with 2 sets of CPU's
                Parminder Gupta

                Clustering would only help you if you have multiple dashboards.

                From what you have said above , it seems you have one large dashboard , clustering would not improve the performance.

                 

                If performance is an issue when you have 64 Cores and 1TB of RAM, you need to look at how the charts and tables in the dashboard have been defined.

                Are you doing complex calculations in the dashboard in multiple charts / tables ? Could these calculations be done on the Database before the data set is extracted ?

                Are you using nexted if statements ? could you replace these with flags ?

                 

                It might not be a server scalablity issue at all ...

      • Re: Server with 2 sets of CPU's
        steven.ahomed

        We are running QV 11.0.11307.0  on two HP ProLiant DL980`s G7 with 8 64-core Intel X7560 processors with  1 TB of RAM. As compared to our version 10 SR5 setup. We are noticing very poor performance opening documents via the  accesspoint using Ajax and slower reload times.  Let me of know of any settings that can be changed to improve performance.

  • Re: Server with 2 sets of CPU's
    Aji Paul

    Karl, Did you figureout the issue?

  • Re: Server with 2 sets of CPU's
    Mohamed Ellougani

    Hi everyone,

     

    I am having same issue, poor perfomance with the following server.

    QlikView Version 11.20 SR2.

     

    IBM HX5 BLADE in COMPLEX Mode

    SSD 50GB RAID 1

    128GB DDR3

    Windows 2008 R2 Enterprise (All fix pack/updated) & Tried Windows 2012 Standard (All fix pack/updated)

    CPU E7-4807 1.86 X 4 CPU (6 Cores) = (24 Cores total) with Hyper threading deactivated

    http://ark.intel.com/products/53569/Intel-Xeon-Processor-E7-4807-18M-Cache-1_86-GHz-4_80-GTs-Intel-QPI

    Simple statistic calculation from a file size of 600MB Test took 4.30 Minutes

     

    Over a less powerful blade it's took 30% less time to achieve same operation.

    IBM HS22 Blade

    SAS 146GB RAID 1

    48GB DDR3

    Windows 2008 R2 Enterprise (All fix pack/updated)

    E5530 2.40 X 2 CPU (4 Cores) = (8 Cores total) with Hyper threading deactivated

    http://ark.intel.com/products/37103/

    Simple statistic calculation from a file size of 600MB Test took 3:06 Minutes

     

    I followed all the possible document online about hardware setting and so on but wasn't able to improve the performance.

    Quick tips #8 - Server Settings For Best Performance

     

    I even tried running a VM on the HX5 over the SAN to rule out the internal SSD drive but that made absolutly no difference at all.

     

     

    Any suggestion will be welcomed.

     

    Thanks in advance!

    • Re: Server with 2 sets of CPU's
      Jason Michaelides

      I recently heard from a reliable source that 4-socket G7s may have a serious performance issue with QV. This could well go for other 4xCPU boxes.  Try halving the CPU affinity in QMC (System > Setup > QVS > Performance) by unticking half the boxes, restart QVS and see how that goes.

    • Re: Server with 2 sets of CPU's
      Gert Gerber

      I had similar experience on SR2.  Performance is much better after I had upgrade to SR3

    • Re: Server with 2 sets of CPU's
      Carlos Lisboa

      Hi,

       

      In a single user scenario it's way better to haver fewer but faster cores than to have more but slower cores.

      The fact that the information has to be split up to be fed to the cores and then joined back always creates an overhead. Overhead results in a performance loss.

       

      In a multiple user scenario that is not a linear conclusion one can draw, but that's usually the case.

       

      Regards,

      Carlos

    • Re: Server with 2 sets of CPU's
      Hampus von Post

      Hi Mohamed,

       

      In your case you have likely created calculations that are single threaded. This could be due to many factors such as

      expressions, data model and so forth.

      When calculations are single threaded QlikView Server (QVS) will only utilize a single core for all or parts of the operation. With 100% single threaded calculation computation time is directly correlated to core clock speed. 

       

      Your previous hardware ran at 2.4 GHz and your newer runs at 1.86 GHz.

      2.4 divided by 1.86 yields about 30% difference, which is exactly what you mention.

       

      So if possible do get high clock speed CPUs, and as always do refer to your local QlikView representative and/or QlikView partner for advice on server recommendations.

       

      The new server is however much more capable at handling more simultaneous users and applications as the theoretical computational capacity is three to four times the previous one, but for this single user single application test your older is faster only due to clock/core speed.

       

      Regards,

      Hampus

      • Re: Server with 2 sets of CPU's
        Mohamed Ellougani

        But I also tried the HX5 in simplex mode same specs wise beside the cpu and I was still getting the same result around 4 minutes. Is that normal?

         

        Sent from my iPhone

      • Re: Server with 2 sets of CPU's
        Mohamed Ellougani

        Mr.Hampus von Post

         

        Thanks for you explanation.

        Can you please tell us why we see activity over all cores when we run the satistic test then?

         

        qlikview.png

        • Re: Server with 2 sets of CPU's
          Hampus von Post

          Hi Mohamed,

           

          Yes there is activity across all cores, so the operation is not single threaded at all times. A calculation done by QVS does not necessarily reside on the same core throughout the calculation. In you case it is still single threaded to a high degree as the overall CPU utilization is low despite having long response times, but all the small sub calculations required could be done by any core sequentially.

           

          Normally with large and single threaded calculations there would be a much clearer indication that a single core is used.

           

          There is an excellent post by Henric Cronström on the calculation engine that's invaluable.

          http://community.qlik.com/blogs/qlikviewdesignblog/2013/08/20/the-calculation-engine

           

          To summarize, with the way your application utilized hardware it is very dependent on single core clock speed.

          I recommend that attention is given to:

          * Does the table have "calculated dimensions",

          * Could the expressions be optimized

          * Is the data model simple

           

          As I stated before refer to QlikView partner and/or QlikView representatives when selecting hardware,

          and if you feel you would like to ask further questions on performance and improvement for your specific document, QlikView Expert Services is great reasource.

           

          Regards,

          Hampus von Post