Security Rules and Performance in Qlik Sense

[Andrew_Delaney]

Security rules are a very powerful tool intended to assist Qlik Sense administrators in managing access to Qlik Sense features. Using these rules administrators can manage not just the ability to access or create applications, but almost all features of the product including data connections, streams and even individual components of the Qlik Management Console.

This allows administrators to define user and administration roles that closely match unique organization needs rather than merely rely on an out of the box user model. Unfortunately, this comes at a cost, when a user logs in and access the system all the relevant security rules need to be evaluated to determine the correct access rights for the current screen. This can potentially lead to long load times and poor performance in especially in large systems.

This blog post will cover a summary of the default user model Qlik Sense ships with, along with a closer examination of how security rules are evaluated and cached and in doing so will provide a framework for administrators to ensure that they can get the most out of their system and where possible reduce unintentional performance impacts on users.

The Default Security Rules

Qlik Sense ships with a set of default security rules that provide a set of administration roles and user roles for organisations to use and modify to meet their own needs.

When you first log in to a fresh installation of Qlik Sense September 2019, you will be greeted with a set of 63 rules which all work together to create that initial user model.

Broadly speaking the model works as follows (Items in bold are unchangeable rules and are always true):

In the Hub:

• Users can create apps
• Owners of items can see those items
• If you can read a stream, you can read all apps published to the stream
• If you can read an app, you can read the content of that app
• If you can update an app, you can update the contents of that app
• If you can read an unpublished app, you can create objects that are part of that app
• If you can read a published app, you can create some objects on that app
• Everyone has access to the Everyone Stream
• Administrators have access to the Monitoring Apps stream

In the QMC:

• The Service Account has access to everything
• The Root Administrator has access to everything
• Audit Administrators have access to audit sections and entities
• Content Administrators have access to content section, entities and connection related security rules
• Deployment Administrators have access to deployment sections and entities, apps (to manage load balancing) and license rules
• Security Administrators have access to security sections and entities

As you can see there are two separate contexts for rules and that, for the most part administrator roles only impact the Qlik Management Console. This is because all administration activity is expected to happen in the QMC and not the hub. It is possible, and in some cases desirable, for your root administrator to operate entirely without an assigned license, though this does prevent them from using the monitoring applications.

One final note about Security rules in Qlik Sense. Rules can only ever add privileges, they can never take them away. This is very important when trying to design your access model, try to think in terms of what users can do, rather than in terms of what they cannot do and the structure of your rules will flow much more easily.

Security Rule Evaluation

Let’s go a little deeper and focus on how the system evaluates security rules.

Every rule has a set of attributes that are used during evaluations:

• Context (Does this rule apply in the QMC, the Hub or both)
• Resource Filter (The parts of the system the rule covers)
• Actions (The privileges the rule grants)
• Conditions (The set of conditions that ultimately determine who is granted the actions)
• A Score (A dynamically generated metric indicating how useful the rule is, the more

When a user accesses Qlik Sense, the system pulls only rules relevant to the current context and resources (for example, when accessing the hub, we need to evaluate the rules for a user’s apps, as well as the rules for streams).

The system also will retrieve the relevant resources (in this case all published applications, applications the user owns, and all streams) to evaluate the combination of rules, user and resources.

It runs through these rules in score order, these scores are stored in the Security Rule Evaluation Cache, which means the values are lost every time the system restarts.

Once it has finished evaluating the rules, the results are stored in the Security Rule Evaluation Cache so that the next time this user accesses the same page the system can simply retrieve the result of the calculation from the cache and if necessary only evaluate the rules on new apps.

The system will exit the evaluation early if the user has already been granted all the requested rights to a resource. For example: the Root Administrator will generally find most of the QMC will load very quickly because the rule evaluation is very simple. One of the oldest rules in any given system grants them all privileges on everything in the QMC, that rule will generally score very highly and so be evaluated very early and then there is no need to perform any further checking as rules can only add and not take away.

On the other hand, a limited user who has been granted access to only the Apps Section of the QMC and a small handful of applications may find that interface takes a very long time to load as the system will first need to evaluate enough rules to give access to the requested resource (i.e. in a system that has no previous evaluation, the less access the user has the longer the evaluation will take.

Finally, security rule evaluation takes place on a mixture of different nodes, some evaluations will take place on the node with the proxy you are connecting to, some will take place on the node with the engine, and some will take place on the central node.

In the case of QMC activity, all evaluations will take place on the central node, and in the case of very limited users, those evaluations will be relatively lengthy and complex. On larger systems this can lead to a rather slow user experience on the QMC. In extreme cases this can lead to the system becoming effectively unavailable on all nodes. In Qlik Sense September 2019 we have implemented 2 feature flags to increase the scalability of the QMC at the cost of reduced functionality.

See How to enable QMC Limits in Qlik Sense (KB 000086580) for details.

The Security Rule Evaluation Cache

In the previous section I mentioned the Security Rule Evaluation Cache Cache, which stores the results of security rule evaluations in order to speed up future access. Why then does your system seem to slow down as more users and apps are added? After all the cache should eventually cover all common system activities.

The cache is deliberately designed to be short lived, there are many circumstances under which it is either partially or wholly invalidated or discarded, meaning that a full rules evaluation needs to be completed once more. This is broadly because we need to know that the cache is correct, anytime something changes (or could have changed) we cannot rely on the cache and must revert to the slower approach.

There are basically 3 types of cache invalidation:

1. Complete invalidation (The entire cache is discarded)
2. User based invalidation
3. Entity based invalidation

1. Complete Invalidation

• Any change to security rules, load balancing or license rules
• Any change to custom properties values (i.e. the potential possible values a custom property can be assigned)
• When the system is restarted (the cache is never written to disk, and rules may have changed since the service last started)
• If the connection to the database is disrupted

As these changes could have a wide ranging impact, the entire cache is discarded and must be rebuilt by user behavior.

When this cache is invalidated, the scores for every rule are also reset to 0, meaning that we also lose any optimizations gained since the session started.

2. User based invalidation

• Any change to a user attribute (e.g. the user is added or removed from a group)
• A user’s custom properties are changed

Rules are often based on user attributes, changes to that user mean we cannot ensure the cached results still apply, so we discard the cached results linked to that user.

3. Entity based invalidation

• Almost any change to a single entity. (e.g. changing an app’s name or owner, approving a community sheet, renaming a data connection)

Much like with users, changing any aspect an entity or resource that can be addressed in a rule means that we cannot ensure the cached results still apply for that entity, so we discard the cached results linked to it.

In Qlik Sense February 2019, additional logging was implemented to record when a cache invalidation occurs along with its type (Complete, User, Entity)

Recommendations/Best Practices:

If you want to provide the best performance for your users try to make rule changes near scheduled downtime, when the cache is already going to be invalidated due to a service restart.

To reduce the number of evaluations that need to take place, try to ensure you have a regular clean up process where you export and delete old apps that are no longer in use.

Limited users in the QMC should also have a very limited scope, i.e. only get access to one or two sections. Where possible try to centralise QMC activities to a smaller number of administrators rather than expand access to the entire user base. In a large organization it is better to have a small number of administrators and a ticket system than to grant every developer access to the QMC for small scale task management.