Seeking Advice/Solution on implementing 3-Tier architecture in SaaS edition of Qliksense

[xy-siang]

Background:

We are trying to create a 3-tier architecture in SAAS edition of Qliksense.

Currently, successfully performed a binary load with the qvf application being stored in one drive.

Problem:

The qvf application that is stored in one drive needs to be updated daily. But as done in Qliksense Enterprise, there is no way we can use the updated data model in SaaS edition of Qliksense.

​Advice / Solution needed:

If anyone has done 3 tier architecture in  SAAS edition of Qliksense and able to automate the update of the qvf application in one drive daily before, could you please help or share the solution for the above-mentioned problem ?

At the same time, is there a way to use any eternal tools to automate the process of file export from SAAS edition of Qliksense?

Appreciate your help! 

Regards, 

XY Siang

[barnabyd]

G'day XY Siang,

I am also attempting to set up a full ETL environment in SaaS. I don't yet have all the answers but I can give you a few of my recently learnings.

1. Use Shared Spaces to store your QVD generator apps and their QVDs 
2. I spoke to a Qlik representative last week and he said that an upcoming release will allow data files to be stored in Managed Spaces as well
3. You can upload some data files using Qlik Data Transfer

I have not yet attempted to use the binary load as I did not like that feature back in the QlikView world. I'd be interest to know if it has changed much.

I hope this is useful information.

Cheers,

Barnaby.

[xy-siang]

Hi Barnaby, 

Thanks for your reply. But let me explained in details so that you will get a clearer picture of the issue.

In QS enterprise on windows:

When performing the 3-Tier architecture,  the binary load in this layer 3  qvf file is referring to the layer 2 qvf file stored in a physical location/server. Whenever the layer 2 qvf file is being updated daily , the layer 3 qvf file will be reloaded at all times, with the latest data shown in the dashboard. 

2. This is done by implementing a task chaining in the following order: 1. Extract -> 2. Data Model -> 3. Dashboard. The 3. Dashboard is referring to the application in 2. Data Model.

In QS SaaS:

Now we are trying to perform the 3-Tier architecture in SaaS just as the same way in QS enterprise on windows as mentioned above.  But there's no such option in SaaS as it was done in QS enterprise on windows.

The problem faced in SaaS:

1. An app is created in SaaS and it's downloaded from SaaS , which was then stored in on-premises. From there, a binary load can be performed.

2. But there's a problem is how can the app be downloaded from SaaS and stored in any location in on-premises on a daily basis in an automated process?

3. I try exploring the qlik datatransfer tool recommended by qlik support team earlier to see if qlik datatransfer can perform the function as mentioned in point 2.

4. Able to use qlik datatransfer to transfer data from on-premises to SaaS, but not from SaaS to on-premises. Can't find options in qlik datatransfer to download the app from Saas to on-premises at any location in an automated process.

Hope that you will have a better understanding of the limitations when trying to  perform the 3-Tier architecture in SaaS of QS.

Regards, 

XY

[Leigh_Kennedy]

Hi,

Check out Qlik-cli.  https://qlik.dev/libraries-and-tools/qlik-cli

This allows you to automate pulling an app from SaaS to on-premise.

Regards.

[barnabyd]

G'day XY,

Thanks for the clarification. I agree that the SaaS hub does not have an option to automatically download an app. I was going to suggest the Qlik Command Line Interface (qlik-cli) but Lkn beat me to it.

Regarding the strategy to download the app in order to reload it, I feel that this is not good architecture. If your data set is large, then you are sending all this data across the wire twice. I would suggest you keep the 'master copy' locally and only upload it after it has been reloaded.

Regarding the usage of binary load to move the data from tier 2 to tier 3, I think that this, also, is not good architecture. I always go for a set of QVDs that make up a star schema. This data is more easily reused in different apps depending on which fact tables and dimension tables are required.

I hope this info is useful.

Cheers,

Barnaby.

[xy-siang]

Hi Barnabyd, 

Qlik support replied that Qlik DataTransfer is a tool to create QVD and ready to use in SaaS environment. Unfortunately, it doesn't export or import app, but it generates QVDs in the on-premise environment for the app in SaaS.

Similar to Lkn, I was advised to try using qlik-cli to export the app automatically. 

Anyway, appreciate your advice given in the above  !

Regards, 

XY 

[dyy]

I am curious why you need the QVF in OneDrive?  What is the use case for not just working with a Qvf in a SaaS shared space?

[dyy]

I would recommend you set up Extract and Transform spaces as follows:

1. Shared Space - Extract for Extract QVF and store Extract QVDs in this spaces Data Files.  Access only by the BI Team

2. Shared Space - Transform for Transform QVF and Transform QVDs in this spaces Data Files , Access to QVF only by the BI TEam and access to  QVDs by Developers with Section Access to limit row level data if needed.  Create a transform space for each group based on access rights to the QVDs only.  

3. Shared Spaces for Developers collaboration on Data Model and UI/UX

4. Managed Spaces for UAT:  Business user testers access

5. Managed Spaces for Production Applications:  General access

[barnabyd]

Thanks @dyy ,

Is there any official Qlik documentation on this architecture with a tutorial and examples? 

Cheers,

Barnaby.