Qlik Community

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
Now Live: Qlik Sense SaaS Simplified Authoring – Analytics Creation for Everyone: READ DETAILS
cancel
Showing results for 
Search instead for 
Did you mean: 
hugmarcel
Specialist
Specialist

qvd read block size

Hi
we have a production and a test server with equal ressources and equal qlikview installation.

Now, on testserver a qvd with 2'000'000 rows is read in 5 seconds (raw optimised), in steps of 300'000 rows.
On prodserver, the same file needs 10seconds for loading (raw optimised), every 150'000 record load is displayed instead.

What could cause this difference?

Thank you - Marcel

9 Replies
hugmarcel
Specialist
Specialist
Author

A thread with a very similar scenario is this one:

https://community.qlik.com/t5/QlikView-Administration/why-is-loading-a-QVD-file-optimized-slower-on-...

No answer is marked as solution.

I have moved the qvd file to different hard discs on either the prod and the testserver. The problem remains the same, dev read has double speed than reading the qvd on the prod server.

I will now turn off Virus scanner on prod and check. Reinstalled QV.exe on prod as well.

Only other thing I find is that MS MDAC versions are different on both servers, but this should not have an impact.

 

eliran
Creator III
Creator III

Hi Marcel,

 

To fully isolate the case, if you got exactly the same resource, same storage, same clock speed, etc, I would check if there are other factors that get in the way.

On prod server, is there a server active? before doing the tests, do you check that you have no other disk activity?

The dev and prod are on the same space? what happends if you move the qvd to a local folder on these servers?

hugmarcel
Specialist
Specialist
Author

Hi

I have already moved the qvd file to local folders, the result remains the
same.

Is there a way to define how many rows will be read from a qvd before the
rowcount is being displayed in the script window?
marcus_sommer
MVP & Luminary
MVP & Luminary

Thinkable are different user-settings in Qlik like generating a log-file and/or writing it instantly. Maybe there are more settings which may have an impact especially if there were adjustments to the easter egg settings. To exclude any influence from them you could remove the settings.ini (after a backup) to see if it makes any differences.

- Marcus  

Or
Champion
Champion

Is the amount of *available* resources identical in both cases? If the production server is busier, this may be an issue where the reason for slower read is having fewer resources available to actually work with vs. a less-busy test server.

hugmarcel
Specialist
Specialist
Author

Hi

I have already deleted the settings.ini ("easter eggs"), but no difference.
The log file settings are the same (local logging in c:\programdata\QlikView\...),
written immediately.

The amount of available ressources (c: / d: disk) is even slightly better on prod.

Yes, there is a QV Server running on both systems. However, if no job (qvb.exe) is running,
still no difference.
Even If a stop the QV Services on prod and disable them, no difference. QV.exe on Test reads qvd files optimized
always double as fast as on prod.

I have stopped running processes on the prod server as possible, still no difference.

The load processes were equally fast on both systems until specific date Feb, 15. Thereafter, out
of a sudden, load speed of both qv.exe and qvb.exe started to be doubled on prod.

 

eliran
Creator III
Creator III

I suggest you should try to run some testing tool, one that integrate cpu and disk.

You should check if it's a general problem, rather then Qlik related problem, before investigating further in Qlik direction.

 

marcus_sommer
MVP & Luminary
MVP & Luminary

Before looking for any technically reasons on the OS + hardware side I suggest to exclude if any possible all thinkable causes on the Qlik side. This means for example:

  • is it definitely the same qvd which is loaded and not different ones which should contain the same content on fields, records and field-values but are independent created
  • are there any other statements within the load-script - I suggest to create a new qvw and using only this single qvd load-statement
  • switching off the logging - especially the write instantly setting - the run-time check could be made with two variable-statements before and after the load
  • checking the run-times with a cmd-line and/or vbs execution
  • which QV releases are included on both side
  • how are the run-times if the qmc performed the tasks

If nothing significantly changes or gives any interesting hints to follow you may trying some more direct load-checks, for example:

  • trying the same with other qvd's - are the results all similar or are there lesser/bigger differences as expected
  • using either qvd's which are smaller as well as much bigger as this one - background for this trial is to see if the run-time differences change in relation to their size/records or rather not
  • adding a where-clause like: where 1=1; to force to break the optimized mode - of course the load will be slower but the run-time differences should be lesser if the storage is the biggest bottleneck
  • making similar checks but not with the qvd's else with a binary load of another qvw
  • probably some more ... which may allow deductions on what might a possible cause or usually more common to exclude certain reasons   

By all trials a close look on the taskmanager or more specialized tools like the resource manager may be also useful to notice any specifics.

- Marcus

hugmarcel
Specialist
Specialist
Author

Thank you, these are good ideas how to test further. I will try out all of them.

Marcel