Qlik Sense Security integration - Security Rules and Section Access

    <Part of http://integration.qlik.com/ and created together with raymond Neves>

     

    Table of Contents

    • Security integration: Provide Single Sign On and share access rights
    • Introduction
      •     Authentication
      •     Authorization using the security rules
    • Detailed security rules
    • Appendix A: Streams
    • Appendix B: Security flow when opening an app
    • Appendix C: Sense ticketing API explained
    • Appendix D: Security approach summary
    • Appendix E: More information and tools about security?
    • Appendix F: Security rule examples

     

    Introduction

    In OEM software integration cases,  users and their authorizations are stored usually in a SQL table and the OEM website already does the authentication step. So the key thing we have to explain you in this article is how do we transfer the user and his rights to Qlik Sense?  The short answer is: We usually do this by using a ticket (token based) mechanism. This ticket can be seen as a passport for the user. It contains his userId, and the groups/roles which define his authorizations.

     

    Make sure you check the introduction and extended security videos first.

     

     

    Authentication

    Before you can use the Qlik Sense system, in most cases you want to know who the user is, we call this authentication.  The next step is that based on this userId you want to assign the user access rights, we call this authorization. Both developers and users communicate with Sense via the Proxy (see this as a webserver which performs the authentication step: who are you?). For each type of authentication we can create a virtual proxy. So normally we would create a virtual proxy for each authentication mechansim your company needs:

    • Internal users using Active Directory
    • External users using ticketing
    • External users using SAML

     

    Authorization using the security rules

    After the authentication (who are you?) you need to think how you want to integrate the authorization (what can you do and see?). The keys you want to protect in Sense are called resources. Example resources are

    • In the client side, called the hub context:
      • Stream
      • App
      • Object.type  inside an app
        • Sheets
        • Stories
        • Dimensions
        • Measures
        • Library (master items)
        • Script
        • Data model viewer
    • In the admin side, called the management console (QMC) context
      • Reload tasks
      • Security configuration
      • Apps and objects
      • Streams
      • Users

     

    There is no mandatory structure you have to follow in Sense. We have designed a very flexible approach in which each "thing" in Sense is a resource. And if you want to "use" it you need to have a "key" that allows you to access that resource. In Sense we protect the resources above with security rules, and it has the following logic:

     

     

    security rule evalutation.png

    In other words:

    • If you want to see a resource (or edit/publish…)
    • You need at least 1 rule that evaluates to true
    • Note: that this rule must provide access (resource filter) to all resources you need. If you want to see a chart, you need access to
      • The stream
      • The app
      • The app.object for at least one type

    Example Admin 1

    • If your Active Directory group = Admin
    • Then you are allowed to access all resources in the context of the QMC

    Example End user 1

    • If your Active Directory group not equal to Admin
    • Then you are allowed to access all resources in the context of the Hub

     

    In most OEM cases we give each customer its own stream,  and therefore we create a rule like this to split the customers in the system

    • Allow a user to view (Action),
    • all dashboards inside a stream (Resource)
    • Provided that the group (e.g. a customer name) of the user matches the name of the stream

     

    houses.png

    Lets illustrate this with the example of a city (a Sense server), it consists of

    • streets (streams) which are used to
    • group houses (apps), and a
    • house consists of rooms (app.objects).

      Most people can only enter 1 house and all rooms. But in order to arrive at your room, you will have to drive though your street, open the house and enter your room. And that is exactly the same way with an OEM security setup in Sense.

     

    SaaS security setup

    Most times you have a SaaS solution with a lot of customers in 1 system. You want to divide this. So, we start with giving each customer its own stream, and once you have access to the stream you are allowed to view all apps. Of course we can limit the access to apps and even sheets, the script and database connections.  This is possible because everything is a resource is Sense. And before a user can access a resource he needs at least one security rule that evaluates to true for the action he would like to do (read, update, publish).

    Minimum requirements of the security rule

     

    Back to our example of the street and the houses:

     

    You always need access to a stream before you can open the app, or view the resources inside an app:

    1. Stream: The highest level is the stream (e.g. a street), once you are in the street you can see
    2. App: The apps (the houses), once you enter a house you can see
    3. App.object: The app.objects (the rooms) like sheets, stories, script, database connections etc.

     

    Import to remember: If you want to see a chart, you always need to create one or more security rules that together provide access to all the above resources. (stream, app, app.object)

     

    Each resources knows always who its parent is. So you can make a security rule that says

    • If the user tries to access a Stream:
      • The name of the resource (in this case a stream) = (must equal)  the name of the group of the user
    • Then: you are allowed to see me

     

    Now the user is authorized to see the stream. The next step is to grant him access to the app and app.object with this rule:

    • If the user tries to access an app or the contents of an app (e.g. sheet)
      • If you have access to my parent (has privilege)
    • Then: you are allowed to see me

     

    Detailed security rules

    Sense security is based on Attribute Based Access Control ABAC. Each time a user requests access to a resource, Qlik Sense evaluates the request against the security rules in the Qlik Sense system. If at least one rule evaluates to True then Qlik Sense will provide the user with access according to the conditions and actions described in the security rule. If no rules evaluate to True then the user will be denied access. The fact that Qlik Sense security rules are property-based makes Qlik Sense very scalable as you can build rules based on properties that apply to groups of users. In most BI systems you need to create a role for each organizational value (Spain, France etc.), this is not needed with Sense. We just use a variable to variable comparison: e.g. If your Active directory group (or group provided by ticket/SAML) matches some property of the stream/dashboard you are allowed access.  For more information about Security rules see this video or the Qlik Sense help

     

    If you don't have a key you won't get in, same like your house. If you have a key you can enter your house, and all rooms inside it. (Access to stream, Yes? Ok, than you might see all the dashboards)

     

    So for Sense, if you have access to a stream maybe you want to show the user all dashboards inside the stream. In this way you keep the security concept very simple and effective. As a starting point I would recommend to give each department (HR, Finance) or customer (Customer A, Customer B) its own stream.

     

    So we can give users access to a dashboard, and define whether he can use/edit/create Sheets, Stories, bookmarks etc. For example,  if the user does not have access, that is a security rule that evaluates to true, he won't see the edit or bookmark button in Sense.

     

    Authorization summary

     

     

    Authorization check points

    Authorization steps:

    Example resources

     

    authorizationCheckPoints.png

    1: Security rules in the QMC: Stream, apps and other resource access control

    1. Which streams are you allowed to see?
    2. Which dashboards are you allowed to see?
    3. Which sheets or buttons (Edit script, edit dashboard, story telling) are you allow to see?
    Apps, sheets, stories, script, database connections

    Nodes

    Content libraries

    Proxies

    Repositories

    Streams

    Users

    Data connections

    Engines

    Extensions

    Virtual proxies

    Schedulers

    Reload tasks

     

    sectionAccess.png

    2: Section access in the script of a QVF to provide row level security

     

    Section access

    This section contains more info about the "second step" of the picture above: Which data and fields are users allowed to see?

     

    Section access to filter data:

    We load your security table as a regular table, we put this in a special section in the script called "section Access". This behaves exactly the same as the normal script. Except that it makes a "mandatory selection" or filter if the user opens the dashboard.  Note that you can load users or groups in this section access column "UserId". If you send in the ticket attribute with the name “group” it will be used to restrict on in section access. So based on the group provided in the ticket you will see only your data.

     

    In the example on the left: if user John logs in, Qlik Sense makes a "fixed selection" which is basically an inner join between the security table and the datamodel. If John logs in, he will only see the data where department is Finance.

     

    Security using hierarchies

    Security is never a simple model, most of the time companies are organized using hierarchies. Qlik Sense allows you do authorize on this.

     

    See this for more info on authorization using a hierarchy.

     

    In this way you can authorize on a node, and the user will see all the children of the node. Note that this demonstrates the power of our script, the authorization on hierarchies is for example not possible with "pure visualizaton tools" which rely on 1 view to be supplied as input for the dashbard.

     

     

     

    Example: OEM  security example use case for a fictive OEM partner

     

    Functional design of the security principles

    • Each customer can only see its own data. Each customer can have 1 or more users. These users can be a member of 1 or more groups/roles which determine what they can see or do in the system. E.g. a manager can see the "whole world" and a regional manager only his "own region".
      • For example we have
        • 2 customers: Sunny and Cloudy Petrol. Each customer can have the roles
          • Manager and
          • Employee.
    • The security concept should be as simple as possible. In the first phase we will authorize on streams and therefore we will give each customer its own stream. (A stream is the key grouping structure of dashboards)

     

    Technical design of Qlik Sense security:

    We have to configure the following layers

    1. Give each customer its own stream.
    2. Create a security rule to authorize on this stream.
    3. Use section access to enforce row level security of the data.

     

    1: A stream per customer

    2: Security Rule

    In the management console of Qlik we define 1 security rule to enable "resource access control".

        • For ticketing use: user.environment.group=resource.name  (in this case the stream is the resource)
        • For AD use: user.group = resource.name

    3:Section access

    In the script we mark a special section as security. In the section we can limit per dashboard what a user/role can see (data filter/reduction). We do this based on the attribute: 'group' you provided in the ticket. In this way you don't need to sync user-group relationships to Sense.

     

    Integrated security overview

    The picture below outlines the complete flow from user single sign on with ticketing, to row level security via section access.

     

    security flow complete.png

     

    Key take aways:

    Authentication

    To create a single sign on model you only need to

    • Create a virtual proxy in Qlik Sense specific for your app
      • Allow only authenticated users
      • Forward the user to your authentication module if not authenticated.
    • Your auth. Module then authenticates the user and requests a ticket for this user, this ticket includes his group membership.
    • Forward the user to the orginal URL he tried to access, with the ticket number appended. (http://server/proxy/hub?ticket=12345)
    • In the Sense QMC, you create 1 security rule to map the name of the stream (grouping of dashboards) to a group you provide in the ticket
    Authorization

    Row level security

    • If you need more fine grained security this is possible, because everything in Sense is a resource and you can assign attributes/tags to each resource where you can authorize on.
    • E.g. authorize on individual app, sheets or buttons
    • Use the group you provide in the ticket also to reduce data in the dashboard (row level security)

     

    So this is what you DON'T need to do anymore

    • You don't need to keep your source system in sync with Qlik Sense. Because the only link between your system and Sense is the group (or other attributes) you assign dynamically in the ticket request code you write. You only need to create the template apps. The rest will be generated by your system by making use of the API's.

     

    More security documentation can be found here

     

    Appendix A: Streams

    See a demo of streams here

    The content in the hub is organized in streams. A stream is a collection of apps that a group of users has specific access to. The users of the stream can have different access rights. Some users might only be able to read the content in the stream, while others might have the rights to publish their content to the stream.

     

     

    Appendix B: Security flow when opening an app

    Introduction

    Users interact with Qlik Sense using authenticated communication through one or more proxies. Information is transferred using cookies. The cookies are necessary regardless of authentication type used. When using a web browser, the web browser sends a ticket to the proxy. The proxy then returns a cookie to the browser and this cookie is included in the communication with the Qlik engine. When using a custom client, the client must set the cookie and provide it to the proxy.

    Authentication

    Authentication is the procedure of verifying the identity and credentials of users wishing to access Qlik Sense. It can be done in various ways:

    • Windows authentication using cookies
    • SAML
    • HTTP header authentication using cookies and headers. Header authentication should only be used when custom authentication systems are used.
    • You can also extend with custom authentication modules like the ticketing method we always use in software integration scenarios.

     

    Security example: The figure below shows the flow in the Qlik Sense security system when a user logs in and opens a app.

    SecurityExampleOpeningAnApp_768x563.png

     

    1. Authentication: The authentication module in the Qlik Sense Proxy Service (QPS) handles the authentication. The credentials provided by the user are verified against information from the identity provider (for example, a directory service such as Microsoft Active Directory).
    2. Session creation: When the user credentials have been successfully verified by the authentication module, a session is created for the user by the session module in the QPS.
    3. Access control system: When the user tries to open an app, the Qlik Sense Engine Service (QES) requests the Qlik Sense Repository Service (QRS) to check if the user is authorized to perform the action. The QRS then checks the repository database, where, among other things, all users and access rights are stored.
    4. The users are imported into the repository database from one or more User Directories (UDs) (for example,Microsoft Active Directory) using Qlik Sense User Directory Connectors (UDCs). The import is triggered by the Qlik Sense Scheduler Service (QSS) and the intervals in-between imports can be scheduled.
    5. Dynamic data reduction: When the user has been successfully authorized by the QRS, the app is opened. Before the data is displayed to the user, the QES performs a dynamic data reduction, where the data that the user is allowed to see is prepared.

     

    See also:

    Planning Qlik Sense Deployments: Authorization

    Planning Qlik Sense Deployments: Access control

    Planning Qlik Sense Deployments: Data reduction

    Managing a Qlik Sense site: Virtual proxies

    Qlik Sense Proxy Service APIs

     

    Introduction to QlikAuth

    You can create the ticketing implementation yourself via the QPS ticket API, or use the QlikAuth module. QlikAuth is a community tool/wrapper for .NET, JAVA, Javascript around the Qlik Sense Ticket API

     

    Appendix D: Security approach summary

     

    mindMapSecurityIntegration.png

    Appendix E: More information and tools about security?

     

    What

    Where

    Security in Sense

    See blogs by Fredrik Lautrup

    Security rule examples

    Section Access

    Tips and tricks for section acess

    Authorization using a Hierarchy

    Complex authorizations

    Data reduction using multiple fields

    Access Control test module

    Using ticketing. This is an authentication module that can be used for testing access control in Qlik Sense. This is also an example of how an authentication module could look in NodeJS

    Session API test module

    This is a example implementation in Nodejs of how the Session API in the Qlik Sense Proxy can be used

    Load Balancing test module

    Proof of concept implementation of the Load Balancing module API that implements a random load balancing algorithm

    Setup header authentication

    Header authentication is nice if you want to perform some tests with tools like fiddler, or if you have an architecture which uses an authenticating reverse proxy. Don't use this in other cases because a malicious user can misuse this technique.

     

        • Using Xrfkey headers, security measure, explains that if you use the REST API (which is an URL with JSON in the body to send messages to Sense), you must put a string XYZ of 16 chars both in the url as well as in the header attributes.

    For testing purposes it is easy to setup a virtual proxy with header authentication. Make sure you use the path for the REST calls which include this proxy. (http://srv/YOURPROXY/qrs). Don't forgot the http:// and ensure you allow http in the virtual proxy.

     

    Example config using Postman

    If you send this:

    Request

    Response from Server

    postman request.png

    postman response.png

    Authentication using certificates

    If you have server side software you can use certificates to authenticate against Qlik Sense. In this way your software tool can talk as admin to Qlik Sense.

    Authentication using ticketing
        • In your software tool you can also supply a ticket you got from the Qlik Sense proxy API.
        • You can use the Qlikauth module from Rikard Braathen for most programming languages
        • See Mindspanks code on Github here

    Export and import the security rules

    Use this tool from Fredrik Lautrup

     

     

    Appendix F: Security rule examples

     

    You can make the security in Sense as complex or simple as you want, I defined a couple of scenario from simple to complex.

     

    Scenario

    Authorization business rule description

    1

    Only authorize on streams

    2

    Authorize on the combination of streams and apps

    3

    Authorize on the combination of streams, apps and resources inside an app (sheets, stories, script, master items)

    4

    Authorize on rows and columns

    20

    An External tool should have full access, except to delete the "Template" and "Everyone" streams and apps.

     

    Scenario 1a: Authorization business rules Stream authorization

    For our fictive company we defined the following rules

    • We have defined groups in the source system. This can be AD groups (user.group) or the groups you supply in the ticket (user.environment.group).
    • Each customer can only see its own stream
      • Within a stream he can see all apps
    • Within an app he can only see the sheets and stories but not the backend

     

    Scenario 1b: QMC Admin

    • Same as 1a
    • But now each customer can also have an admin that should have access to the QMC

     

    Scenario 2: Authorization business rules: App authorization

    • Each customer can only see its own stream
      • Within a stream he can see specific apps
    • Within an app he can only see the sheets and stories but not the backend

     

    Scenario 3: Authorization business rules: Sheet authorization

    • Each customer can only see its own stream
      • Within a stream he can see specific apps
    • Within an app he can only see the sheets and stories but not the backend

     

    Scenario 4: Apply row/column level filtering

    • Each user can only see the data in the app where he is authorized for
    • For example, an employee can only see the orders (and everything associated) he created.

     

    Security rule code for each scenario

    Implementation of scenario 1a using the QMC - Security rules

     

    Before we start, remember that If you want to see something, you need one or more security rules that give you access to

    • The stream
    • The app
    • The app.object

     

    So in the examples below we create 3 rules (to make it manageble we splitted the rules per resource type, you can also create 1 big rule)

    • A rule for the stream
    • A rule for the app
    • A rule for the app.object

     

    Modify standard rule set

    Also we have to look at the standard security rule set in Qlik Sense. Here we have to disable some rules to prevent our your have too much authorizations. You can always find the rules that are active for your stream/app by going to the audit. Double click a cell, and you can see why John, has the rights to create app.objects (sheets, or basiscally all the "things" inside an app)

     

    The rules below have to be disabled. To offer too much access to our users for now.

    security rules disabled.png

     

    Access to resource

    Description

    Security rule condition

    Stream

    Example security rule to give each user access to its stream

    For use with Ticketing

    user.environment.group = resource.name

     

    For use with Microsoft Active Directory

    user.group = resource.name

    App

    Example security rule to give

    • each user access to all underlying apps (for a consumer/display only user)
    • if it already has access to the stream.
    (resource.resourcetype = "App" and resource.stream.HasPrivilege("read"))

    App Object

    Summary

    Example security rule to give each user access to

    • all underlying objects, but not the script and the data model viewer
    • if it already has access to the app.

    Detailed

     

    • If you a trying to "touch" a resource that is inside an app (app.object means sheets, stories, script)
      • You are allow to see these all items inside an app as long as they are
        • Published
        • Not the script and the loadmodel,
        • And you have access to the stream where the app is in.
    ((resource.resourcetype = "App.Object" and

    resource.published ="true" and

    resource.objectType != "app_appscript" and

    resource.objectType != "loadmodel") and resource.app.stream.HasPrivilege("read"))

     

    If you want to go one step further you can also authorize on sheets:

    • Manager can see all sheets
    • Employees can not see the "finance dashboard" and the "HR dashboard"

    The security rule for employee:

    (resource.resourcetype = "App" and resource.stream.HasPrivilege("read")) or ((resource.resourcetype = "App.Object" and resource.published ="true" and (resource.objectType= "sheet") and (resource.name!="Finance Dashboard" and resource.name!="HR Dashboard") ) and resource.app.stream.HasPrivilege("read"))

    Script: Section access

    Row level security: In section access you can use the group of the ticket or AD to ensure row level security. In this way you can make sure that someone who is member of "group: Germany", can only see results in the dashboard where the data belongs to Germany.

    Use the script below to authorize on groups (supplied in the ticket or via AD etc.) and on individual users

    sectionAccess groups.png

     

    Note: normally you just load this table from your SaaS platform. For demo purposes I just do an inline load.

     

    Template Rule

     

    (resource.resourcetype = "App" and resource.stream.HasPrivilege("read")) or ((resource.resourcetype = "App.Object" and resource.published ="true" and resource.app.stream.HasPrivilege("read"))

     

     

    Implementation of scenario 1b: QMC admin per customer

    Access to resource

    Description

    Security rule condition

    Apps and Stream

    An admin of a specific customer is allowed to see their own stream and apps in the QMC

    Filter: QmcSection_Stream, QmcSection_App*, Stream*, App*


    For use with Ticketing

    user.environment.group="admin"

    and

    resource.stream.HasPrivilege("read")

     

     

    For use with Microsoft Active Directory

    user.group="admin"

    and

    resource.stream.HasPrivilege("read")

    If you then create a user in your source system, and give him the admin group. He is able to see the QMC, only for his stuff.

    And he can see this:

    QMC Admin screenshot.png

     

    Only his stream and only his apps which are published in his own stream

     

     

    Read access but not a specific stream and its apps

    Read access but not a specific stream and its apps.png

     

    View everything in the QMC except apps and streams

     

    View everything in the QMC except apps and streams.png

    Scenario 30: Audit display all

     

    Audit display all.png

     

    Security rule summary

     

    See Qlik help for an overview of the options in the security rules. In my multi tenant SaaS demo platform I used the rules below.

     

    Rule name

    Condition

    Resource filter

    Actions

    Description

    Z_OEM_API_TOOLS_APP

    ((resource.stream.name!="Templates" and resource.stream.name!="Everyone" and user.roles="QRSMeteorAdmin" and resource.IsOwned() and resource.owner = user))

    App_*,App.Object_*

    All

    Root all for apps, except the everyone and template stream apps

    Z_OEM_API_TOOLS_QMC

    ((user.roles="QRSMeteorAdmin" and resource.resourcetype!="App" and resource.resourcetype!="Stream"))

    *

    All

    Everything in the QMC except the Streams and APPS

    Z_OEM_APP_ACCESS

    resource.stream.HasPrivilege("read") or

    (resource.resourcetype = "App.Object" and resource.published ="true" and resource.app.HasPrivilege("read"))

    App_*,App.Object_*

    Read

    see all apps if you have access to the stream

    Z_OEM_STREAM_ACCESS

    ((resource.name=user.environment.group or (resource.name="Everyone") ))

    Stream_*

    Read

    end user: if you see the stream you are allowed to see all apps for your customer

    Z_OEM_CONTRIBUTOR

    user.environment.group="Contributor"

    and  resource.published ="false" and

    resource.app.HasPrivilege("read")

    App.Object_*

    Create, update

    A user with group contributor is allowed to create sheets in exiting apps, but no app creation

    Z_OEM_DEVELOPER

    user.environment.group = "Developer"

    and

    ((resource.owner = user and resource.stream.Empty())

    or

    (resource.app.HasPrivilege("read") and resource.published ="false" ))

    App*

    All

    A user with group developer is allowed to create apps, sheets and load data.

    Z_QMC_ADMINS_CUSTOMER

    ((user.environment.group="Admin"))

    QmcSection_Stream, QmcSection_App*

    Read

    QMC Menu only. Only give access to stream if the name of the stream match the AD group. Note, we disabled the default Stream Rule

    Z_QMC_ADMINS_CUSTOMER2

    user.environment.group = "Admin" and ((resource.resourcetype = "App" and resource.HasPrivilege("read")) or (resource.resourcetype = "App.Object" and resource.app.HasPrivilege("read")) )

    App_*

    Edit, publish

    Administrators are only allowed to maintain the apps and app.object in the QMC

    Z_Root_ALL

    ((user.roles="RootAdmin"))

    *

    all

    Root: Hub and QMC full access