Best Practices of Qlik Sense Multi-Node Setup, Load Balancing & Maintenance(Let's Discuss Here)

[rohitk1609]

Hello Everyone,

First of all, thanks for reading! I found it is amazing to connect myself to people around the world and communicate with my writings.

This page is indented to discuss best practices for  Multi-Node Setup & Load Balancing primarily, You may drop your use case or problem in comment, we will provide you best possible solution instantly.

Below content guide you to set up Multi-Node Qlik Setup & Load Balancing. It will help you to know Qlik technical terms, ask question more precious way which will help usor anyoe to provide solution more accurately.

Business intelligence is an agile process, gradually BI solution changes and updates. Enterprise grows so are data and end users.

When data grows and business logic gets complex, we need more resources to calculate the result and serve to users. Afterwards, Single Node (small) site performance decreases with time which could lay down the credibility and quality of BI solution.

Multi node site is  Qlik site(could have multiple Qlik servers ) where load is getting managed between more than one system and could have more than one  Fai-lover Candidate which takes charge in absence of central node automatically.

Note: we can have multiple fail-over candidates in one site, we call it passive central.

There are certain parameter which help you to take decision whether we should move to multi node site or remain on single node.

Qlik Sense Enterprise on Windows on-premises Deployment Scenarios

So below content is for such users who have made mind to upgrade their Qlik site to multi node.

Few naming conventions we will follow in below content:

1. Consumer node: which must has proxy service enable
2. Scheduler node: which has scheduler and engine services are enabled
3. Central: which is controlling whole Qlik site.
4. Failover Candidate(Passive central)
5. Resources: RAM and Cores(virtual cores)

Since June 2017 , installation setup has shared persistence way only where Apps and RepoDB data stay at persistence layer and both are shared with other nodes (rim or child).

Now why do we need to share Apps and Repository data with child or rim nodes?

Answer: to manage load in simple

Next Question: what exactly we share to balance the load practically?

Answer: Resources in Qlik site.

Next Question: what will be achieved if we share the resources?

Answer: Reduce the resource consumption while end users are accessing apps from one node i.e. central to consumer node (proxy service is must) and reduce the resource consumption and distribute it to scheduler node (scheduler and engine service is must) while any app reloads.

There could be many scenario how you would design your multi node site:

1. You want one central and one consumer and one scheduler node.
2. Two scheduler and two consumer and central
3. Central and scheduler node.

Ideally: Two scheduler and two consumer and central node comes in best practice for extra large enterprise.

You can read about single node, small, medium , large and extra large Qlik shared site in detail:

Qlik Sense Enterprise on Windows on-premises Deployment Scenarios

Multi node site would increase your Qlik Sense site availability, how?

We have a concept fail-over candidate, which is one of the scheduler node or such node which has all services and take charge automatically while Central node is down. We can manipulate the sensing time of failover candidate to central node in latest version.

Solutions which will guide you to deply multi node side

1. How to Install multi node site:

Installation of Multi Node site

Installing Qlik Sense in a multi node site - Qlik Sense

1. How to configure multi-node site:

Installing Qlik Sense in a multi-node site

       2.  Load balancing:

Adding load balancing

Configuring load balancing to isolate development nodes

          3. Fail-over candidate

Fail- candidate or we can call it passive central node which is ping in a set time(you can customize it below 10 minutes) to central or master node and when it doesn't get response it will take charge and become active central or master node.

Note: Where you install your postgresDB during installation that server has one extra service Qlik Repository Database which is running the postgresDB. So in case you don't have a dedicated persistence layer and installed Repo DB on central node(any server which has Qlik Sense too) then if your central node get down due to power failure or blue screen then fail-over node won't take charge. 

Primary condition of fail over concept needs QlikSenseRepositoryDatabase to be run all time. if QlikSenseRepositoryDatabase is down then whole site is down. 

One important application is attached to article which helps you to review your Qlik site. If you  have any concern over Qlik repository size, QLogs DB size or any component which is accessing excess size than expected, then download the app, import to your Qlik Sense, open it on HUB, there will be three connection which come along with application i.e. PostgreSQL_postgres, PostgreSQL_QLogs and PostgreSQL_QSR. Update password with your super user password of postgresDB which you entered during installation and save it. 

If your user name added as postfix after updatation of connections , go to QMC and make it as they were earlier.

Please do above activities by RDP not from end user side. Reload the app and share it with me so I can review it.

After utilizing above solutions, please post your queries in comment section what is your use case i.e. how many servers are available, how your user comes to site, network load balancer is available or not, reloads are executing on scheduler nodes etc.

In general, multi node or load balancing concept is vast and vary with client to client, so I do believe you may ask question much better when you understand the right terms of Qlik, went through above articles and requirements are clear.

Consider this resource as a open form where we all can post our queries on load balancing and multi node site.

There are few other documents which would help you in Qlik administration

1.Dynamic Sheet Exception With Stream and App Level Security

2.Sheet or App Object Level Security Qlik Sense

Reach to me if there is need of any clarification or need assistance with kumar.rohit1609@gmail.com

Follow my profile on LinkedInhttps://in.linkedin.com/pub/rohit-kumar/2b/a15/67b,

Add yourself to a great community of Qlik People on Facbook 

Add yourself to Qlik intellectual people group on Facebook:

https://www.facebook.com/QlikIntellectuals

When applicable please mark the appropriate replies as ACCEPT AS SOLUTION and LIKE it. This will help community members and Qlik Employees know which discussions have already been addressed and have a possible known solution. Please mark threads as LIKE if the provided solution is helpful to the problem, but does not necessarily solve the indicated problem. You can mark multiple threads as LIKE if you feel additional info is useful to others.

Thanks,
Rohit Kumar

[m_perreault]

Hi Rohit,

Is it possible to create Load Balancing rules that are user based opposed to stream/app based.  For example can I create a rule that users are only able to access Rim Node 1 and not Rim Node 2?

Additionally is it correct to say that each rim node will have a unique url?

Thanks,
Mark

[rohitk1609]

Hi,

you can create load balancing rule but I don't think it will balance the load between proxies or RIM nodes. for try purpose, I created one load balancing rule where resource is User_asddcfgvvkdhykjshye and node is node1 but when I am opening any app on hub it is not bounding the apps for user whose id is asddcfgvvkdhykjshye to node1. it is working as per apps or stream rules.

Confusing point is when you create such rule no error comes out and no preview happens

but I can say, my answer is NO for user level load balancing rule. 

Each RIM node who has proxy service enabled has a separate URL. but it depends how your users traffic come to you RIM nodes or proxy  nodes. are you using network load balancer  to bring users ?

[axel]

Hi!

We are in the process of adding another node to make a multi-node installation from our current single-node.

The reason we want a second node is not that the load is to high, but that there are resources that we cannot access from the central node that we want to connect to.

There is a firewall between the central node and the rim node, but we have opened all ports we could find information about in the documentation.

The new node is also in a different domain, in another AD, than the first node.

However, on the new node scheduler service and proxy service won't start.

Before we invest to much time in this I'd like to ask if you know if it is supported having machines from different ADs and domains in the same cluster.

Thanks!

[rohitk1609]

User from different domain could work if user from other domain has required access.

Few questions for you

1. Can you ping central node server from rim node server by Hostname of central node server ?

2. the services user on rim node has full access of Share Folder ?If you have RDP of rim node, can you reach to share folder with service user of rim node \\centralserver\sharefolder

3. After installation, is certificates distributed in Node section ?

4. did you verify ports connectivity by powershell or telnet ?

Thanks,

Rohit

[axel]

Hello Rohit!

Thank you for your quick response!

1. Yes, ping is possible

2. Service user on the rim node is the same as on the central node, and access using Windows Explorer works as intended when using the FQDN of the central node. Therefore we made sure the FQDN was included in the path for the shared folders in the postgresql database. It is actually the servers that are part of different AD:s

3. Yes, certificates have been distributed using the instructions from qmc. It is possible to add the node in qmc, but it says the scheduler service has stopped.

4. The ports have been tested, but since we are not going to run this node as a proxy we have not opened 80 and 443. Could this be an issue for the scheduler service?

Thanks!

Axel

[rohitk1609]

Few more questions,

1. did you add your service user in log on as service in group policies , might be name is same but due to different domain or machine its directory is different ?

2. did you restarted the rim node after installation, if no please do it once?

3. Did you tested all the ports in Overview section in https://help.qlik.com/en-US/sense/November2018/Subsystems/PlanningQlikSenseDeployments/Content/Sense...

See this seems a connectivity issue, so the first point is check ports and especially check on both side, inbound and outbound.

Thanks,

Rohit

[axel]

Thanks!

1. Several of the qlik services start, it's just the proxy and scheduler service that do not start, but since this node is not configured to be a proxy that might be why that service doesn't even start. On the server the service user is in the local admin group and the qlik-specific group mentioned in the documentation. However if we have an issue with the firewall, authentication requests might still fail.

2. Node is restarted

3. We will test all the ports using telnet to verify that the communication works. Is it ok to keep 80 and 443 closed it if the node is not a proxy node? I will post here if the issue was in the firewall.

Thanks again for your help!

[axel]

Hi again,

I have another, related question.

When we get this node up and running as a part of the cluster I have a specific data connection that will only work if it runs on the rim node. I have other data connections that will only work if they run on the central node.

I have looked through the documentation and posts here, but I have not yet really found the info I am looking for, so I will ask here:

Is it possible to run just a specific reload task on a specific node, every time? This node will only be a reload node, not proxy.

Best regards,

Axel

[rohitk1609]

Please find my inputs below:

1. For reloads or execute any app on rim node you have to make run scheduler and engine service. Repo is mandatory.

2.  Auth of exe. file installation , log on as service and port communication are completely different things. Qlik needs ports to be opened even if it is working stand alone. 

3. If possible start with all mandatory ports required , for proxy too, if you are intended to make your rim node as failover candidate , you have to run all services at rim node.

4. try to make rim node as failover candidate. its a great feature and makes your Qlik site highly available.

5. Yes it is definitely possible to bound any task or any app to reload on rim node only. you can set which reload will execute on which node.

You can control it by load balancing rules.

To investigate the connectivity issue between two Qlik servers, the first point is PORTS.

Keep positing your updates to fix it asap.

Thanks,

Rohit

[npmurray]

Hi,

I'm in the process of make a number of changes to our Qlik setup following a review that we got Qlik to do our environment.  They made a number of recommendations;

1.  Move the Repository DB to its own Server
2. Create a cluster file share
3. Add addtional engine nodes for capacity

We have completed items 1 & 2 and are working on item 3.

My quesiton is.  We have create the new Engine node Server, installaed Qlik, added it via the QMC, create the proxy and virtual proxy.  So now that I have done all that what config changes are needed to load balance the trafiic over both engine nodes?

Many thanks