Best Practices of Qlik Sense Multi-Node Setup, Load Balancing & Maintenance(Let's Discuss Here)

[rohitk1609]

Hello Everyone,

First of all, thanks for reading! I found it is amazing to connect myself to people around the world and communicate with my writings.

This page is indented to discuss best practices for  Multi-Node Setup & Load Balancing primarily, You may drop your use case or problem in comment, we will provide you best possible solution instantly.

Below content guide you to set up Multi-Node Qlik Setup & Load Balancing. It will help you to know Qlik technical terms, ask question more precious way which will help usor anyoe to provide solution more accurately.

Business intelligence is an agile process, gradually BI solution changes and updates. Enterprise grows so are data and end users.

When data grows and business logic gets complex, we need more resources to calculate the result and serve to users. Afterwards, Single Node (small) site performance decreases with time which could lay down the credibility and quality of BI solution.

Multi node site is  Qlik site(could have multiple Qlik servers ) where load is getting managed between more than one system and could have more than one  Fai-lover Candidate which takes charge in absence of central node automatically.

Note: we can have multiple fail-over candidates in one site, we call it passive central.

There are certain parameter which help you to take decision whether we should move to multi node site or remain on single node.

https://help.qlik.com/en-US/sense/June2018/Subsystems/PlanningQlikSenseDeployments/Content/Deploymen...

So below content is for such users who have made mind to upgrade their Qlik site to multi node.

Few naming conventions we will follow in below content:

1. Consumer node: which must has proxy service enable
2. Scheduler node: which has scheduler and engine services are enabled
3. Central: which is controlling whole Qlik site.
4. Failover Candidate(Passive central)
5. Resources: RAM and Cores(virtual cores)

Since June 2017 , installation setup has shared persistence way only where Apps and RepoDB data stay at persistence layer and both are shared with other nodes (rim or child).

Now why do we need to share Apps and Repository data with child or rim nodes?

Answer: to manage load in simple

Next Question: what exactly we share to balance the load practically?

Answer: Resources in Qlik site.

Next Question: what will be achieved if we share the resources?

Answer: Reduce the resource consumption while end users are accessing apps from one node i.e. central to consumer node (proxy service is must) and reduce the resource consumption and distribute it to scheduler node (scheduler and engine service is must) while any app reloads.

There could be many scenario how you would design your multi node site:

1. You want one central and one consumer and one scheduler node.
2. Two scheduler and two consumer and central
3. Central and scheduler node.

Ideally: Two scheduler and two consumer and central node comes in best practice for extra large enterprise.

You can read about single node, small, medium , large and extra large Qlik shared site in detail:

https://help.qlik.com/en-US/sense/February2019/Subsystems/PlanningQlikSenseDeployments/Content/Sense...

Multi node site would increase your Qlik Sense site availability, how?

We have a concept fail-over candidate, which is one of the scheduler node or such node which has all services and take charge automatically while Central node is down. We can manipulate the sensing time of failover candidate to central node in latest version.

Solutions which will guide you to deply multi node side

1. How to Install multi node site:

Installation of Multi Node site

Installing Qlik Sense in a multi node site - Qlik Sense

Qlik Sense September 2017 - Installing Qlik Sense in a multi-node site

1. How to configure multi-node site:

         Configure a multinode installatio

       2.  Load balancing:

      Adding load balancing

       Load balancing

       Configuring load balancing to isolate development nodes

          3. Fail-over candidate

Fail- candidate or we can call it passive central node which is ping in a set time(you can customize it below 10 minutes) to central or master node and when it doesn't get response it will take charge and become active central or master node.

Note: Where you install your postgresDB during installation that server has one extra service Qlik Repository Database which is running the postgresDB. So in case you don't have a dedicated persistence layer and installed Repo DB on central node(any server which has Qlik Sense too) then if your central node get down due to power failure or blue screen then fail-over node won't take charge. 

Primary condition of fail over concept needs QlikSenseRepositoryDatabase to be run all time. if QlikSenseRepositoryDatabase is down then whole site is down. 

One important application is attached to article which helps you to review your Qlik site. If you  have any concern over Qlik repository size, QLogs DB size or any component which is accessing excess size than expected, then download the app, import to your Qlik Sense, open it on HUB, there will be three connection which come along with application i.e. PostgreSQL_postgres, PostgreSQL_QLogs and PostgreSQL_QSR. Update password with your super user password of postgresDB which you entered during installation and save it. 

If your user name added as postfix after updatation of connections , go to QMC and make it as they were earlier.

Please do above activities by RDP not from end user side. Reload the app and share it with me so I can review it.

After utilizing above solutions, please post your queries in comment section what is your use case i.e. how many servers are available, how your user comes to site, network load balancer is available or not, reloads are executing on scheduler nodes etc.

In general, multi node or load balancing concept is vast and vary with client to client, so I do believe you may ask question much better when you understand the right terms of Qlik, went through above articles and requirements are clear.

Consider this resource as a open form where we all can post our queries on load balancing and multi node site.

There are few other documents which would help you in Qlik administration

1.Dynamic Sheet Exception With Stream and App Level Security

2.Sheet or App Object Level Security Qlik Sense

Reach to me if there is need of any clarification or need assistance with kumar.rohit1609@gmail.com

Follow my profile on LinkedInhttps://in.linkedin.com/pub/rohit-kumar/2b/a15/67b,

Add yourself to a great community of Qlik People on Facbook 

Add yourself to Qlik intellectual people group on Facebook:

https://www.facebook.com/QlikIntellectuals

When applicable please mark the appropriate replies as ACCEPT AS SOLUTION and LIKE it. This will help community members and Qlik Employees know which discussions have already been addressed and have a possible known solution. Please mark threads as LIKE if the provided solution is helpful to the problem, but does not necessarily solve the indicated problem. You can mark multiple threads as LIKE if you feel additional info is useful to others.

Thanks,
Rohit Kumar

[rohitk1609]

Hi,

Thanks for reaching to us.

If I  understood your concern correctly, load balance depends on two things

1. How you have added you node in QMC i.e services like engine , proxy

2. your load balancing rule.

Few questions for you:

1. Are you using network load balancer ? 

2.  how many total nodes in your Qlik site after adding new one ?

3. How you have planned to balance the load or distribute the load among nodes like you wana put reloads on one node and let use access apps on other node,

what it means, when you execute any app it will consume resources(RAM and CPU) from one Qlik machine and when user will open any app it will consume the resouce from other Qlik server.

Few people let central node free (1 Central, 1 scheduler and 1 Proxy) and put all reloads on one and let user consume apps by one proxy node.

in general we call proxy node as consumer node.

you don't need to activate proxy service on scheduler node but engine is must.

Load balance concept people use  to solve a problem, so explain me in detail what you are trying to achieve ?

Thanks,

Rohit

[npmurray]

Thanks for the response.

No, we are not using a network load balancer and have no plans to use one.

After adding this new node, we will have 3, 1 central, 2 engine (consumer) nodes.  As well as a separate Repository DB server and clustered File Share.  We are planning to add a separate proxy node so by the end well have

1 Proxy node
1 Central/Scheduler node
2 Consumer nodes for user access
1 Consume node for development
1 Repository DB Server
1 Windows Clustered File share
1 NPRinting Server

We are looking to balance the load from users on the 2 consumer nodes an seperate out developmet on another node.

[rohitk1609]

@npmurray

let me add my points so you can align with me.

Consider the image I am attaching in this coment.

1. Server 1, which your persistence layer, in Qlik language , where we have postgreDB and apps , we call it persistence layer. Very few firm has separate persistence layer, which is a good thing or I can say recommended by Qlik. 

Generally people make that server  a persistence server which is highly available that other servers so if your all server at one place or on cloud then I can say try to utilize this server resourcces in daily use like make it your central node or scheduler node.

NOTE: ALL SCHEDULER NODE OR FAILOVER CANDIATE ARE ALMOST EQUAL TO CENTRAL NODE EXCEPT SCHEDULER STATUS. 

Since June 2017 we have flip flop central node concept where, your central node will be fliped to any other failover candiate when your current central or master is unavailable.

2. If you have no NLB it means your IT will make user traffic to your proxy node , my question is why do you have a dedicate proxy server, is that server will be at any other location ?

3.  You can make repo DB and file share as one server, why are you using two servers separately ?

4. Yes I am with N printing on separate server .

See you are doing exactly what Qlik recomends  but Qlik recommendation are based on some things.

Qlik fileshare should be on highly available server but if you are repoDB and Qlik file share are on same cloud cluster or location then why you split it to two machine ?

Please consider Fail-over candidate on high priority, I missed about it in your comments. You are gonna architect so big site and it is no worth without failover   concept.

You can achieve  load balancing rule solution on

https://help.qlik.com/en-US/sense/November2018/Subsystems/ManagementConsole/Content/Sense_QMC/config...

You can mail me to communicate better kumar.rohit1609@gmail.com

Thanks,

Rohit

[axel]

Hi again!

We have now made the also the scheduler service work inside the DMZ. We opened these ports (all TCP connections checked with telent):

business_server is the central node and dmz_server is the reload node.

• business_server <- UPD 137 -> dmz_server
• business_server <- UPD 138 -> dmz_server
• business_server <- UPD 4748 dmz_server
• business_server <- TCP 80 dmz_server
• business_server <- TCP 139 -> dmz_server
• business_server <- TCP 443 dmz_server
• business_server <- TCP 445 -> dmz_server
• business_server <- TCP 4239-> dmz_server
• business_server <- TCP 4242-> dmz_server
• business_server <- TCP 4243 -> dmz_server
• business_server <- TCP 4432 dmz_server
• business_server <- TCP 4444 -> dmz_server
• business_server <- TCP 4747 -> dmz_server
• business_server <- TCP 5050 dmz_server
• business_server <- TCP 5151 dmz_server

However, this did not allow us to start the scheduler service. It looked like it wanted to access the Internet, which is not allowed from the DMZ. Against best practices we took the server out of the DMZ for a reinstallation. With the internet access all services started as expected, and it continued to work when the server was moved into the DMZ and internet access was removed. Do you know if the scheduler really requires internet access to be activated?

Thanks for helping us out!

Kind regards,

Axel

[axel]

Hi!

I still haven't completely solved my problem but since I now have the servers working I fell I am moving forward a bit. My current problem is that I have the central node which should display all applications and I have a separate reload node that should reload only one application. This application needs a data connection to a database that can only be reached from the reload node.

• I assume I cannot create the data connection and thus not the application on the central node, since the central node cannot reach the database. Is this correct?
• Thus I need a load balancing rule that allows me to connect to the reload node (which must also then have the proxy service running). I want to connect directly to the reload node; I don't want any connections to the reload node to be forwarded (by a load balancing rule) from the central node to the reload node. It looks like I still need some kind of load balancing rule for this. Right now when I try to connect I get the message "The proxy cannot perform load balancing on any of the engine services. Reload browser or contact system administrator". Can this be solved by a load balancing rule? If so, how?
• How do I make the reload task run on only the reload node using load balancing rules?

Thanks a lot for any solutions or pointers you can give!

Kind regards,

Axel

[rohitk1609]

Hi Axel, 

Few points which will help you:

1. If I understood your Qlik site, you have two servers ,lets say A which is in DMZ zone, means outside of your network security and one which is non DMZ lets say B which is inside of security. My suggestion is make server B as your central node or master node and use server A to let user access your apps.

2. I do not think scheduler node needs internet, it might happen when you enable internet it opens certain ports which scheduler node needs to connect to master node.

3. If you wana put all your reloads on scheduler node , then go to QMC=>scheduler=> central= set it to master only, then no load will happen on central or master type.

4.  In case if you make DMZ  server as central then yes you can make connecton to database by taking RDP on scheudler node, login as https://localhost/hub and make connection. Please enable all services while adding  scheudler node to central

please confirm DMZ mode or server is which is outside or security and publicly available ? 

Thanks,

Rohit

[axel]

Hi Rohit,

1. I was unclear about the network structure, sorry; it looks like this:

https://docs.google.com/drawings/d/1LqBIGVrmJEfKdmAfCrhtBBt42QVwkS0RZ6hDt6aGSXw/edit?usp=sharing

2. During installation we had the reload node in the business network and the installation was successful. After moving the server to the DMZ the scheduler kept working. When we tried installing everything in the DMZ without internet the scheduler service would not start and we could see blocked connection attempts to a few places on the internet. This might all be a coincidence though, so we can leave it as it is.

3. Since the node in the business network needs to reload a lot of data sources that cannot be reached from the protected network DMZ, and since this is the node that will serve the apps to all users, it has to be the central node, and it has to be both master and slave scheduler.

4. Connecting to a terminal server in the DMZ allowed me to run the browser in the same network and connect to the DMZ Qlik server.

Thus what I am now facing seems to be to correctly configure Qlik to allow me to

• Create an application that I can work with on the reload
• Make this application visible on the central
• Create a data connection to Oracle on the reload
• Make reloads happen on the reload node for only this application

I been experimenting a little, so I have been able to create the application and make it visible on both nodes, but when creating a data connection on the reload node my options are much more limited than on the central node, for instance the Oracle connection wizard is not there, which is a problem.

I will not write what rules I have implemented, as it was only experimentation and it is probably pretty far from best practice.

So, if I could get some ideas on how to best solve the above problems I'd be very grateful!

Sincerely,
Axel

[rohitk1609]

Hi Axel,

Sorry my friend for late reply!

Please find my points w.r.t. your points:

1. As I read DMZ zone is which is open for internet and available for external users but as you said  your scheduler server is protected one and under DMZ zone. you are calling it reload node, fine.

You opened few ports for oracle and few to connect but I wana add a point here, Qlik has a large list of internal ports too which it needs to run smoothly. It might happen your server is installed but you never know which feature will not work unless you investigate and find it.

2. Qlik download and installs .net frame work as per its version so it might happen it was looking for internet to do it. Its very common client don't offer internet on Qlik server but Qlik server runs fine.

3. if you are calling server which is in business network CENTRAL so I hope repository and QlikShare folder is on same server. Now you open QMC or hub from any of server I hope you get same list of apps.

 You can open HUB locally on reload node and create connection with oracle and make sure your all reload happens on this node, IN CASE IF YOU MAKE YOUR CENTRAL SERVER MASTER AND SLAVE , YOUR RELOADS WILL FAIL.

4. Make sure your service user can access share folder from reload server.

When you say  "Make reloads happen on the reload node for only this application"  how you did that ?

5. Did you make reload server as failover candidate. if no what will happen if your central is down ?

6. "Oracle connection wizard is not there,"  this is not at Central node ?

I am not sure what extacly your problem is but please make sure all possible aspects you covered.

mail me in case you need urgent help: kumar.rohit1609@gmail.com

Thanks,

Rohit

[shiveshsingh]

Hi

I have applications installed one one Qliksense server.

Now as per requirement, i need to migrate all the data to one SAN drive and other servers.

There will be one central node, one HA node for central, 1 Scheduler Node, 1 Scheduler Node HA, 3 Consumer nodes.

How can i proceed with the requirement, ?

I have looked on backing up the Qliksense site but will all the current data including repository will be stored in new SAN drive?

[rohitk1609]

Hi Shivesh,

If your concern is to backup your Qlik site:

1.Backup QSR first https://help.qlik.com/en-US/sense/September2018/Subsystems/PlanningQlikSenseDeployments/Content/Sens...

2.  Backup QlikShare folder where your apps, archived logs and content resides.

3. Backup your all Qvds and external file which your Qlik apps or QMC uses .

Then store these things to any drive, and with help of all these things you can restore your site.

Thanks,

Rohit