Qlik Community

Qlik Scalability

A group dedicated to advancing Qlik scalability and performance. In this group we will share knowledge on how to perform load tests of Qlik products.

naveenbalanagu
New Contributor III

Performance Testing in a Multi Node environment?

I am running the load scripts using Scalability tools. The test/production environment is setup the following way.

  • Qlik Center Server
  • Proxy1
  • Proxy2
  • Engine1
  • Engine2
  • Scheduler1
  • Scheduler 2

Since there are two proxy servers (for load balancing), do I need to point my scripts to Qlik central or is it OK to point to any one of the two proxy servers?

I've tried using proxy1, proxy 2 and central and I am able to access the apps/hub using any of them. Can anyone tell me what is the best way to configure the scripts to simulate more accurate user scenario?

Does the services installed on these servers matter in test scenarios?

1 Solution

Accepted Solutions
Employee
Employee

Re: Performance Testing in a Multi Node environment?

Hi,

You should simulate how the users would end up in the system. So if users would be evenly spread to each proxy node then split the simulated users evenly between the proxies. If you have a virtual proxy that are for specific users (accessing with header for example) then all users doing that need to use that virtual proxy.

That's basically what I can say without knowing the details, from you description it seems like you want to use 1/3 of the users to each of the proxy nodes (assuming central has a proxy service as well), but I might have misunderstood.

As for services installed matter, I'm not sure I follow the question. Everything installed on servers involved will matter for the test as well as production. The scalability tools will simulate virtual users but those will use access the system properly.

Regards

Sebastian

View solution in original post

7 Replies
Employee
Employee

Re: Performance Testing in a Multi Node environment?

Hi,

You should simulate how the users would end up in the system. So if users would be evenly spread to each proxy node then split the simulated users evenly between the proxies. If you have a virtual proxy that are for specific users (accessing with header for example) then all users doing that need to use that virtual proxy.

That's basically what I can say without knowing the details, from you description it seems like you want to use 1/3 of the users to each of the proxy nodes (assuming central has a proxy service as well), but I might have misunderstood.

As for services installed matter, I'm not sure I follow the question. Everything installed on servers involved will matter for the test as well as production. The scalability tools will simulate virtual users but those will use access the system properly.

Regards

Sebastian

View solution in original post

naveenbalanagu
New Contributor III

Re: Performance Testing in a Multi Node environment?

Thank you for the explanation. I am a newbie in performance testing and also I don't know how exactly the load balancing works. What I came to know from my infra team is that - the incoming requests will be randomly/evenly distributed between proxy 1 and proxy 2 (using another enterprise level proxy to route the requests. This is because we have mashups loaded in a web app). But for my test activities (which is to check performance when users directly connect to QlikSense hub to access the apps (this is for another users group in the organization) then they can use either of the proxy servers (by directly entering the hub URL). So I am planning to run multiple scenarios to check the response time when


All users using

1. Only Proxy 1

2. Only Proxy 2

3. Combination of both


(So I believe the default load balancing rules and the load balancing servers added in virtual proxy has nothing much to do as these are only in case of a fail over.)

Regarding services installed on these machines - I see all important services  (repository, engine and proxy) in windows services but enabled/disabled as below.

   

Server NameProxy ServiceRepository ServiceEngine Service
QLIKCENTRALYesYesYes
QLIKPXY1YesYesNo
QLIKPXY2YesYesNo
QLIKENG1NoYesYes
QLIKENG2NoYesYes

When I run the scripts, I am planning to collect the logs for all the above servers (no matter which proxy I am pointing the scripts).

Thank you again for your patience for answering my questions.

naveenbalanagu
New Contributor III

Re: Performance Testing in a Multi Node environment?

Hi Sebastian,

I ran the scripts as mentioned above. The response time and RAM consumption seems to be fine but the CPU utilization is high for one of the servers.

Here is the screenshot of the test reports ( System Resource Utilization - By Server sheet in results app)

capture.JPG

Can you please review this and tell me if we have to improve the app or upgrade the server Hardware. I am trying to benchmark the results based on the information provided in https://oem.qlik.com/media/DS-Technical-Brief-QIX-Engine-Memory-Management-and-CPU-Utilization-EN.pd...

So the services are using above 70% of the RAM (in all servers) which is as expected and also I don't see any spike in consumption during the test.

RAM

capture.JPG

CPU utilization and scaling over cores

But average CPU utilization is > 70% in one of the servers.

capture.JPG

The engine service is running in (Eng 1, Eng 2 and Central servers).

Here is the response time graphs and attached file has the server information. Thank you in advance

capture.JPG

Employee
Employee

Re: Performance Testing in a Multi Node environment?

Hi,

First of all it is very hard to give advice as we don't know details about the app and script, However I can give a few pointers. Looking at CPU, the system is indeed saturated. It could be caused by a very large app or "bad" app design. It could also be environment that is too small. Lastly, the script simulating load might be too aggressive.

You are correct that it is generally good using 70% of RAM, as this will mostly be consumed by engine cache. However in this case there is very little available RAM per node so it is not hard to reach this amount. The RAM spent on cache will be what remains from RAM used by Operating System, other services and also the initital RAM footprint of just loading the app into RAM. If your app have a large RAM footprint then very little of will be left for cache.


It looks like the environment is an E7-8880 with total of 64 hypertheraded cores and 40GB RAM that have been virtualized into several smaller parts. It could be that you would get better performance keeping it as one virtual machine. One reason is to reduce the overhead of the operating system for instance.


A good advice I can give is to manually open the app and do some selections to observe how ram and CPU behaves. Also try the app out on a different environment if possible. That can give hints on wheter or not your deployment is the issue or if the app is too large or can be designed in a better way. 

You can also try the app evaluator that is included in the scalability tool package.


Regards

Sebastian

naveenbalanagu
New Contributor III

Re: Performance Testing in a Multi Node environment?

Thank you so much Sebastian. I know it would be difficult to find out the facts with out full details about the app and the script. But your analysis is very useful for me to proceed further. I will run some more scripts today with reduced load and check if that makes any difference.

Yes, I am opening the hub in browser while the test is running to see the response time.  I will try app evaluator as well.

Before we close this thread, one final question - Is there any way we can check the amount of RAM consumed by each app?

Thanks again for all your help..

Employee
Employee

Re: Performance Testing in a Multi Node environment?

Hi,

Yes, if you restart the services and note down the RAM consumption before and after opening the app you will know how much is required to load it into memory. If you continue with selections while being the only user in the system you will see how much RAM and CPU is required to do the calculations you trigger.

Regards

Sebastian

Highlighted
naveenbalanagu
New Contributor III

Re: Performance Testing in a Multi Node environment?

Thank you.