QMC data reload Performance Issue

[#1]

Hi,

I am facing an scenario where database is oracle , tables count is 150 and data size is around 500 GB, Now, I am executing it 4 times is or data refresh cycle is 4 times in day but each time it takes like 15 to 20 min to reload or refresh, Can anyone help me how to Increase the performance , less data refresh time ,

Point here is I am looking for an approach , how to start analyzing , ETL part, Modeling, App creation, So I will have a way to follow to increase performance ...

This is a very common requirement comes by client

[vlad_komarov]

Please check here (https://help.qlik.com/en-US/sense/2.2/Subsystems/Hub/Content/Scripting/ScriptRegularStatements/Binar...), but the short version:

1. You need to add a Binary Statement to your script:

Binary [lib://Qlik_Apps_Connector/a4484de1-fud1-4954-b5t5-911100000000];

Please note that it should be the VERY FIRST line in your script (before SET definitions)

2. You need to use a App's ID (a4484de1-fud1-4954-b5t5-911100000000), not QVF file name there.

3. You can get this ID when open the app in the browser:

That's it.

Regards,

Vlad

[#3]

Hi Vlad,

It is giving me an error of Data connection not found (Qlik_Apps_Connector). I think I need to create a new Data connection first with administrator login because when we install Qlik Sense server with administrator credentials the folder only accessed to Administrator where all the application saves same as in Qlik Sense Desktop ,where applications saves in

Documents=Qlik=>Apps.

Currently I am logged in with Non Administrator User.

Please confirm Do I need to apply some permission as when Binary load will work , It can access the data connection made by administrator ?

[vlad_komarov]

Hi Rohit,

Yes, you need to create your new connector (Folder type) to the directory where your applications are located.

I've used a Qlik_Apps_Connector connector as an example.

And yes, it's recommended to use a admin account to create this connector.

Vlad

[#5]

Hi Vlad,

Thanks for your response.

I have created a new Folder connection with name Qlik_Apps_Connector but when I saved it the user name administrator comes with connection name like Qlik_Apps_Connector(Domain_administrator). then I logged out and logged in with USER1 and then name of connection become Qlik_Apps_Connector(Domain_administrator)(Domain_USER1) . Can you please explain why this is happening and for your knowledge USER1 has root admin rights. what id USER1 doesn't has root admin rights ?

[vlad_komarov]

Rohit,

If I understand you correctly, the Connector's name is updated as you log-in as different user.

Could you please provide a screenshot for that?

It should not happened. You can create a Connector with Admin User account, share it with all users and it should do it.

Here is my system:

As you can see the connector uses the original owner's name.

Here is how you control other users' access:

Make sure you give full access to Admins and System accounts (red rectangles)

Just curious: did you try to edit the first connector created by Domain_administrator account from Domain_USER1 account?

VK

[#7]

Hi Vald,

I am attching the snapshot below:

Domain: UIC

Administrator Name : Administrator

User Name: Rohitk

Note:Rohitk has root admin access. Do I need to care about the permission of data connection for Rohitk ?

Secondly , I am not sure for your last point (Just curious: did you try to edit the first connector created by Domain_administrator account from Domain_USER1 account?), May be I did because the connection with name Qlik_Apps_Connector(UIC_Administrator)  [Initally showing when i logged with Rohitk account] ]was not working initially.

Third: Once i have tried to create a connection by Rohitk but when I tried to run the script with same connection it was failing , Can you please add your notes on this too.

Please explain in little explain so I can pick it with step by step

[vlad_komarov]

I never saw the case when the connector name was updated like you are showing.

You probably did something unusual to cause that...

But it should not really matter: you should be able to use this connector from any account you've granted access for.

Just re-create a connector from Admin account and try to use it (without modifying it from other accounts)

Based on your screenshot you gave Admin and yourself just a Read and Update access. Why not a full access (see my screenshots above).

Did you give an access to other people?

What kind of error(s) you are receiving while executing the script? What's your "binary load" statement?

You can attach a logs here so the issue will be more clear.

VK

[#9]

Hi Vlad,

I am logged in with a user which has root admin role granted. But I couldn't find this window which you mentioned above . Can you Please help me to know how you reached there ? What I understand whenever I create a new connection by Admin account same time I need to grant access to others no matter what role they has, no matter root admin. Please correct me if i am wrong

[vlad_komarov]

Here are steps:

In "Data Connections" (1) select your connector, click on "Edit" (2)

Find "Associated Items --> User Access " (3).

Use "Security Rules" option (4) to edit individual rules..

Regards,

Vlad

[#11]

Hi Vlad,

Hope you are doing good!

I got the issue what exactly is ,In your screen User access option is coming and in my window User option is coming.

Please help me to find why in my account User access is not coming , even mine user has Root Admin Rights.

Where can be issue is ,Did i updated any rule by mistake ?

I am attaching a snap shot and coming option is highlighted.

in yellow