Qlik Community

Ask a Question

Documents

Qlik Sense documentation and resources.

Announcements
Our May 2021 end-to-end product release from Data Integration to Data Analytics is out! READ DETAILS

Configuring AWS S3 Blob Storage in Qlik SaaS (Qlik Cloud Services, Qlik Sense Business)

Levi_Turner
Employee
Employee

Configuring AWS S3 Blob Storage in Qlik SaaS (Qlik Cloud Services, Qlik Sense Business)

Objective:


For this document, we will review how to setup a connection to an AWS S3 bucket using Qlik SaaS (Qlik Cloud Services and Qlik Sense Business).


AWS S3 Setup:

  • Navigate to the S3 Service and create a bucket

aws-1.png

  • Specify a name (which will need to be unique, globally), select the desired region.

aws-2.png

 

 

 

  • Note: It is highly recommended to leave the default Block all public access setting. Qlik SaaS does not require a public bucket for use and use of a public bucket is considered extremely risky from a data security perspective.


AWS IAM Configuration

Since Qlik SaaS uses IAM user accounts to connect to S3, we will create an IAM user who will have full control over the S3 bucket. An optional configuration will be outlined later which will create an IAM user who has only read rights to the bucket. This shows the ability of Qlik SaaS to inherit granular IAM rights to buckets. For example, creation of read and read / write users who are scoped to departments is possible an encouraged in Qlik SaaS.

  • Navigate to the IAM Service in the AWS Console
  • Create a user. The Access Type should be set to Programmatic access so that the account has an access key and secret key.

aws-4.png

 

 

 

  • We will apply permissions at the bucket level, so continue through Permissions
  • Tags are optional so configure if used in your AWS tenant
  • Copy both the Access Key and Secret Key

aws-5.png

 

  • Navigate to the newly created user’s record in IAM and copy the User ARN

aws-6.png

 

 

 

AWS S3 Configuration

  • Navigate to the Properties panel of the S3 bucket and copy the bucket's ARN

aws-7.png

 

 

  • Navigate to the S3 bucket’s Permissions configuration and edit the Bucket Policy. An example broad policy which permits full access to all files in the bucket can be adapted from this schema:
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": [
                  "arn:aws:iam::<myIAMuser>"
              ]
          },
          "Action": "s3:ListBucket",
          "Resource": "arn:aws:s3:::<myS3Bucket>"
      },
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": "arn:aws:iam::<myIAMuser>"
          },
          "Action": [
              "s3:GetObject",
              "s3:PutObject"
          ],
          "Resource": "arn:aws:s3:::<myS3Bucket>/*"
      }
  ]
}

 

Qlik SaaS Configuration


In Qlik SaaS, users can create data connections in two places: inside of a Qlik app or in the Hub. For this guide, we will use the Hub option.

  • Navigate to the desired space for the data connection, select the gear icon, and select Data sources

aws-8.png

 

 

  • Scroll to Data Connections and add a data connection
  • Select Amazon S3
  • Enter the Access Key and Secret key for the IAM user along with the appropriate bucket name and AWS region

aws-9.png

 


Alternative Policy Configuration

As mentioned above, with AWS's Policy flexibility, we can create custom policies on the bucket which give a low level of granular control over which user(s) can do which actions on which files.

To provide two examples of this flexibility:

Option 1 - Read-Only vs. Read/Write Access

This policy provides for separation of read and read/write activities:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": [
                  "arn:aws:iam::<myReadUser>",
                  "arn:aws:iam::<myReadWriteUser>"
              ]
          },
          "Action": "s3:ListBucket",
          "Resource": "arn:aws:s3:::<myS3Bucket>"
      },
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": "arn:aws:iam::<myReadWriteUser>"
          },
          "Action": [
              "s3:GetObject",
              "s3:PutObject"
          ],
          "Resource": "arn:aws:s3:::<myS3Bucket>/*"
      },
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": "arn:aws:iam::<myReadUser>"
          },
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::<myS3Bucket>/*"
      }
  ]
}

 

Option 2 - Root User vs. Read-Only Line of Business / Project / Department 

This policy provides for a read-only user who is scoped to purely a specific "folder" for a department / project / line of business:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": [
                  "arn:aws:iam::<myReadOnlyDepartmentUser>",
                  "arn:aws:iam::<myRootUser>"
              ]
          },
          "Action": "s3:ListBucket",
          "Resource": "arn:aws:s3:::<myS3Bucket>"
      },
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": "arn:aws:iam::<myRootUser>"
          },
          "Action": [
              "s3:GetObject",
              "s3:PutObject"
          ],
          "Resource": "arn:aws:s3:::<myS3Bucket>/*"
      },
      {
          "Effect": "Allow",
          "Principal": {
              "AWS": "arn:aws:iam::<myReadOnlyDepartmentUser>"
          },
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::<myS3Bucket>/<myDepartmentFolder>/*"
      }
  ]
}

 

 

Version history
Revision #:
1 of 1
Last update:
‎2021-01-19 04:47 PM
Updated by:
 
Contributors