topic Re: How much data goes over the network connection? in Connectivity & Data Prep

How much data goes over the network connection?

xudo — Tue, 09 Apr 2024 14:23:04 GMT

I'm trying to determine how much data goes over the network before it is written to QVD. Since the QVD format is higly compressed, just looking at the size of the QVD's on disc is a not a good measure for this.

In the example here below in the printscreen I created an Excel file of 34.913 44 kB, the QVD size of this file is 3.842 kB.

In the log of the distribution service for this app on this location: "C:\ProgramData\QlikTech\DistributionService\1\Log\20240409\140313 - ADMIN - Testfile" I see a log entry "Memory Allocation Delta for this file=34.11MB".

Am I right to assume that with an OLEDB data source, this is the amount of data that has to be sent over the network?

Re: How much data goes over the network connection?

Anil_Babu_Samineni — Wed, 10 Apr 2024 05:04:31 GMT

@xudo This is only logical memory that is consumed, But there will be physical memory that it consumes during storing of QVD. My best approach to use process monitor such cases to get accurate in windows (Critical : If you run process monitor in windows server, It consume lot of memory of the server, so please test when there is no load on server by users and immediately disable that to avoid slowness for users).

Re: How much data goes over the network connection?

marcus_sommer — Wed, 10 Apr 2024 10:47:24 GMT

The Qlik internal RAM consumption is irrelevant to the network-traffic of loading the data and storing them again. I never tried to monitor it but I would think it's quite the same as the file-sizes of the sources + targets.