Qlik Community

Technology Partners Ecosystem Documentation

Qlik Technology Partners Ecosystem

Qlik Sense Enterprise for Elastic

Qlik Sense Enterprise for Elastic

Qlik Sense Enterprise for Elastic

Qlik Sense Enterprise for Elastic (QSEfE) is built for the Cloud and deployed in Kubernetes-managed [Linux] Containers. It [currently] provides dynamically scalable capacity for consuming Qlik documents that have been reloaded elsewhere. Those documents are distributed into QSEfE using the Multi-Cloud capability documented earlier.

https://help.qlik.com/en-US/sense/latest/Subsystems/PlanningQlikSenseDeployments/Content/Sense_Deplo...

The components of Qlik Sense Enterprise for Elastic can be grouped into four classes:

 Get very familiar with Kubernetes Best Practices!

  • Do not expose K8S management plane ports externally unless protected
  • Limit the blast radius of a compromised container
  • Prohibit the MITM reading of sensitive data transmitted to and from containers
  • Protect Service API

See also https://www.cisecurity.org/benchmark/kubernetes/

Single-Node Deployment

A Single-Node deployment of Qlik Sense Enterprise for Elastic serves very little purpose other than familiarizing with the Kubernetes tools. The presence of the Repository and Persistence Store on the single node obstructs scaling out the deployment to multiple Nodes, as each would have unintendedly different content.QE-SN-01.png

 

 

 

 

 

 

 

 

The single-node deployment may however be useful for ensuring that you have selected and configured an appropriate Load Balancer, that authentication to the Identity Provider is working properly, that you can use Qlik Multi-Cloud to distribute a Qlik Sense Document from Qlik Sense Enterprise for Windows into Qlik Sense Enterprise for Elastic and that you can access the Qlik content via the Elastic Hub.

After this basic validation, you need to relocate the Persistence Store and MongoDB to separate Nodes before scaling out Qlik Sense over multiple Kubernetes-managed Nodes.QE-SN-02.png

 

 

 

 

 

 

 

The MongoDB repository can be replaced with your own managed instance of MongoDB, mLab (recently acquired by MongoDB) or MongoDB Atlas (available as a managed service within GCP).

The Persistence Store is implemented in an NFS FileShare. You can supply an NFS FileServer in your own instance of a Google Compute Engine (with attached persistent storage) or use the BETA Google Cloud FileStore service.

QE-SN-03.png

Multi-Node Deployment

Kubernetes can be configured to monitor resource utilization [using Prometheus] and automatically re-scale Qlik Sense over more Pods (called Horizontal Pod Autoscaling (HPA)) if CPU/RAM exceed a threshold, or quiesce and scale down Pods which have no active Qlik Sessions on them.

QE-MN-01.png

Take care to ensure that the Kubernetes environment is appropriately secured from inappropriate or malicious administration.

Labels (1)
Comments
Partner
Partner

Hi Michael, excellent post! I'm working on a Qlik Sense Enterprise install on Kubernetes and your document helped tremendously.

I had a few questions though:

1. Is each node pod container basically a single node Qlik Sense cluster? i.e. it has all necessary services installed, QRS, QES, etc?

2. In your intro, you mentioned that this allows for 'consumption of Qlik documents', so is this only meant for production use, with no app creation? Is it possible to have a 'dev' container similar to other multi-node deployments? Or would you suggest having a separate Kubernetes instance for dev, UAT, and Prod?

 

Thanks again

 

0 Likes

I really ought to update this content to refer to Qlik Sense Enterprise for Kubernetes rather than the older Elastic branding.  There's a link to slightly more relevant content at the bottom of  https://community.qlik.com/t5/Technology-Partners-Ecosystem/Getting-Started-with-Google-Cloud-Platf... 

See also https://community.qlik.com/t5/Qlik-Sense-Enterprise-Documents/Qlik-Sense-Enterprise-on-Kubernetes-Su...

The older Elastic product supported only consumption of Qlik Sense documents, which had to be reloaded using Qlik Sense Enterprise for Windows then Multi-Cloud distributed to the Elastic deployment. The newer Kubernetes product supports authoring and consumption without a Windows component, but is still a little limited regarding supported Identity Providers (authentication) and missing some companion products (eg GeoAnalytics, NPrinting, InsightBot, Qlik Sense Mobile) but that is roadmap activity! 

Although some customers/partners are delighted that Qlik Sense Enterprise for Kubernetes removes a dependency on Microsoft Windows, the goal of this product is to achieve dynamic scalability across potentially enormous clusters in order to service Enterprise users. 

With appropriately configured scaling rules in Kubernetes, you could dynamically provide authoring and consumption capacity within the same instance of Qlik Sense Enterprise for Kubernetes. That configuration occurs at the Kubernetes level rather than within the equivalent of the Qlik Management Console. Ultimately this will be a matter of how you want to perform Content Lifecycle Administration, and you might instead have a UAT instance of Qlik Sense in another Kubernetes Namespace or Kubernetes Cluster.

Basic familiarity with Kubernetes can be achieved using single-node Minikube (yes, the single node can contain all required Qlik services/pods) before progressing to a managed cluster on a Cloud provider such as Google's GKE, Microsoft or Amazon's Kubernetes capabilities - though on-Prem Kubernetes on Bare Metal is also fine (albeit adventurous). If experimenting with scaling out across multiple nodes, take care to ensure that the Persistence filesystem and the MongoDB are instantiated outside the cluster, and check that the Load Balancer in front of the Ingress pods (Hub & QMC) supports websockets satisfactorily. 

0 Likes
Partner
Partner

Thanks Michael, I've been looking through the additional documentation you shared but I'm still not clear on how the Qlik nodes within the elastic component (may be called something else now) relate to each other.

Does this still maintain a central vs rim relationship or do they act as standalone nodes, coming into play only when a certain RAM threshold has been reached? If it's the latter, how would we set up a failsafe node in case of any issues?

0 Likes

In the Kubernetes world a pod self-registers and discovers the location of a related service by interacting with the IPtables or DNS on the Kubernetes Master Node. As far as I understand at the moment, the Qlik services/pods coordinate via stateful data in the MongoDB.

You can scale Qlik Sense over more Kubernetes Nodes in a linear fashion without having to "pin" any of the services or functionality to particular nodes (they're simply running "somewhere" in the cluster).  Any particular Service may consist of many Pods, that may be scaled out (ideally over multiple nodes in the Kubernetes cluster, but potentially on the same Node) using configuration values in your "values.yaml" file based on the template/master values that can be observed in the TGZ application itself, downloadable directly from the Helm Repository at https://qlik.bintray.com/stable 

Auto-Scaling particular services over more pods is relatively advanced configuration that I don't have public documentation for yet. One can easily imagine that we won't need many Hub Ingress pods but may require more Engine pods. This scaling can be automated based on eg CPU or RAM utilisation.  Scaling out is not so difficult or delicate as scaling back when utilisation later reduces - this requires some consideration otherwise you may end up with idle pods deployed on many Kubernetes Nodes. 

Version history
Revision #:
3 of 3
Last update:
‎2019-01-16 08:50 AM
Updated by: