Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.

Connectivity & Data Prep

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now

Who Me Too'd this topic

piotrek_w
Contributor III
Contributor III

Data Gateway - File (via Direct Access gateway) – slow, bad performance

Hi All,

 

We would like to use File (via Direct Access gateway) however it seems to be significantly slower than fetching data from QVD file that resides on our Tenant.

For 2GB QVD file we need 3 to 4 more time to fetch the same file – in scenario when only one file is fetching a the same time. In the case we have 2 files loading in parallel then we need up to 5 times more to fetch this file.

According to Network Activity in Resource monitor the send speed for ConnectorAgent.exe is about 10-12 M B/sec – roughly 80 mpbs.

Questions

  1. Do you have similar experience? Is it normal? Can someone provide some statistics from other environments?
  2. Does the gateway have any shaping or maximum transfer speeds?
  3. Is there anything that we can change in our current set-up to boost the transfer speed?
  4. The connectoragent.exe gets an internal IPV6 address for the job, is this ok? Or does it require/prefer ipv4?

Test environment set-up.

  • Qlik Gateway host server: Intel®Xenon® Gold 6140 CPU @ 2.30GHz, 512 GB RAM - located in Sweden
  • Qlik Tenant: eu.qlikcloud.com
  • Test QVD file: Transactions_9; approx. 2GB
  • The connectoragent.exe gets an internal IPV6 address for the job
  • Microsoft .NET Framework 4.7
  • Microsoft .NET Runtime – 8.0.6
  • Gateway version 1.7.1
  • According to our local IT there is no WPAD – they used winhttp show proxy command to check it:piotrek_w_0-1739525811117.png

     

Test Results:

1.Uploading QVD file directly to Qlik cloud (from gateway host server) – web browser: Edge:

Send B/sec ~ 26M

piotrek_w_1-1739525877436.png

2. Load test QVD file that was uploaded and saved previously on our tenant – we needed 60 sec to load file:

piotrek_w_2-1739525925614.png

3. Uploading the same QVD file via Data Gatway.

Send B/sec ~ 10M – we need more than 200 sec to upload file:

piotrek_w_3-1739525977131.png

piotrek_w_4-1739526026868.png

4. Uploading 2 files in parallel: 

I created a copy of the test QVD (Transactions_10) file and run two different scripts to check what happen when we would like to read this data in parallel

piotrek_w_5-1739526236814.png

piotrek_w_6-1739526244551.png

When I run these scripts almost in the same time I can see only one ConnectorAgent sending data ~ 11 M send b/sec:

piotrek_w_7-1739526349176.png

When I run second script 10 sec later I can see that we have to ConnectorAgents but the total speed is similar like in the first case...

piotrek_w_8-1739526423977.png

 

Thanks in advance!

BR,

Piotr

 

Labels (2)
Who Me Too'd this topic