topic Re: qvd read block size in QlikView

qvd read block size

hugmarcel — Fri, 05 Mar 2021 11:21:12 GMT

Hi
we have a production and a test server with equal ressources and equal qlikview installation.

Now, on testserver a qvd with 2'000'000 rows is read in 5 seconds (raw optimised), in steps of 300'000 rows.
On prodserver, the same file needs 10seconds for loading (raw optimised), every 150'000 record load is displayed instead.

What could cause this difference?

Thank you - Marcel

Re: qvd read block size

hugmarcel — Sun, 07 Mar 2021 13:21:39 GMT

A thread with a very similar scenario is this one:

https://community.qlik.com/t5/QlikView-Administration/why-is-loading-a-QVD-file-optimized-slower-on-prod-then-on-dev/td-p/1160235

No answer is marked as solution.

I have moved the qvd file to different hard discs on either the prod and the testserver. The problem remains the same, dev read has double speed than reading the qvd on the prod server.

I will now turn off Virus scanner on prod and check. Reinstalled QV.exe on prod as well.

Only other thing I find is that MS MDAC versions are different on both servers, but this should not have an impact.

Re: qvd read block size

eliran — Sun, 07 Mar 2021 15:05:11 GMT

Hi Marcel,

To fully isolate the case, if you got exactly the same resource, same storage, same clock speed, etc, I would check if there are other factors that get in the way.

On prod server, is there a server active? before doing the tests, do you check that you have no other disk activity?

The dev and prod are on the same space? what happends if you move the qvd to a local folder on these servers?

Re: qvd read block size

hugmarcel — Sun, 07 Mar 2021 19:37:36 GMT

Hi

I have already moved the qvd file to local folders, the result remains the
same.

Is there a way to define how many rows will be read from a qvd before the
rowcount is being displayed in the script window?

Re: qvd read block size

marcus_sommer — Mon, 08 Mar 2021 11:32:14 GMT

Thinkable are different user-settings in Qlik like generating a log-file and/or writing it instantly. Maybe there are more settings which may have an impact especially if there were adjustments to the easter egg settings. To exclude any influence from them you could remove the settings.ini (after a backup) to see if it makes any differences.

- Marcus

Re: qvd read block size

Or — Mon, 08 Mar 2021 12:06:20 GMT

Is the amount of *available* resources identical in both cases? If the production server is busier, this may be an issue where the reason for slower read is having fewer resources available to actually work with vs. a less-busy test server.

Re: qvd read block size

hugmarcel — Mon, 08 Mar 2021 16:21:26 GMT

I have already deleted the settings.ini ("easter eggs"), but no difference.
The log file settings are the same (local logging in c:\programdata\QlikView\...),
written immediately.

The amount of available ressources (c: / d: disk) is even slightly better on prod.

Yes, there is a QV Server running on both systems. However, if no job (qvb.exe) is running,
still no difference.
Even If a stop the QV Services on prod and disable them, no difference. QV.exe on Test reads qvd files optimized
always double as fast as on prod.

I have stopped running processes on the prod server as possible, still no difference.

The load processes were equally fast on both systems until specific date Feb, 15. Thereafter, out
of a sudden, load speed of both qv.exe and qvb.exe started to be doubled on prod.

Re: qvd read block size

eliran — Tue, 09 Mar 2021 06:39:46 GMT

I suggest you should try to run some testing tool, one that integrate cpu and disk.

You should check if it's a general problem, rather then Qlik related problem, before investigating further in Qlik direction.

Re: qvd read block size

marcus_sommer — Tue, 09 Mar 2021 09:05:07 GMT

Before looking for any technically reasons on the OS + hardware side I suggest to exclude if any possible all thinkable causes on the Qlik side. This means for example:

is it definitely the same qvd which is loaded and not different ones which should contain the same content on fields, records and field-values but are independent created
are there any other statements within the load-script - I suggest to create a new qvw and using only this single qvd load-statement
switching off the logging - especially the write instantly setting - the run-time check could be made with two variable-statements before and after the load
checking the run-times with a cmd-line and/or vbs execution
which QV releases are included on both side
how are the run-times if the qmc performed the tasks

If nothing significantly changes or gives any interesting hints to follow you may trying some more direct load-checks, for example:

trying the same with other qvd's - are the results all similar or are there lesser/bigger differences as expected
using either qvd's which are smaller as well as much bigger as this one - background for this trial is to see if the run-time differences change in relation to their size/records or rather not
adding a where-clause like: where 1=1; to force to break the optimized mode - of course the load will be slower but the run-time differences should be lesser if the storage is the biggest bottleneck
making similar checks but not with the qvd's else with a binary load of another qvw
probably some more ... which may allow deductions on what might a possible cause or usually more common to exclude certain reasons

By all trials a close look on the taskmanager or more specialized tools like the resource manager may be also useful to notice any specifics.

- Marcus

Re: qvd read block size

hugmarcel — Tue, 09 Mar 2021 20:49:40 GMT

Thank you, these are good ideas how to test further. I will try out all of them.

Marcel