Pre caching document on Access Point: automated way to open a doc after distribution so first user doesn't have to wait for it to cache?

[#1]

Is there a way (in Publisher, via macros, batch commands or windows Task manager) to "pre-cache" a QV document on the Access Point once it has been re-distributed?

We have a large application (360M+ records, up to 19 GB in memory) which takes several minutes to open on the Access Point once it has been reloaded and distributed. The long wait is not only inconvenient but also leads to users attempting to close the IE browser during the 'open' process which causes an error.

We'd like to somehow pre-cache the application on Access Point so the first user to open it doesn't experience this delay. Ideas?!

We have the "Always Available", "Always Loaded", "Pre-Load" options set, so the document is technically a "loaded document", but it certainly is not cached.

We are running QV 8.50.6326 developer and server with 64-bit, 128 GB 16 processor server and use IE Plugin.

Thanks! Gracias!

[vgutkovsky]

Tyler,

I was also under the impression that preloading would cache the document in server memory. At least it works in 9.0. If this is not happening in 8.5 for some reason, one solution could be to just open the document at night inside the server using command line automation. Would this work for you?

Regards,

[#3]

Thanks for a prompt response.

Does opening the document on the server (the doc located in the servermountedQVW folder) with a command line (qv.exe docname.qvw) cache the document for the Access Point users? Do I need to Open in Server or something special?

I guess I'm not sure exactly where and how the caching takes place for a document hosted on the Access Point (QV Server). Does this question make sense?

[vgutkovsky]

The question makes sense and it's a good one. I honestly don't know. To be on the safe side, you can make the command line statement a URL that will open the QVW. For instance, if you have a file called test.qvw, your command line statement could be

"c:\program files\internet explorer\iexplore.exe" qvp://localhost/test.qvw

. This way, you will make sure that the file opens in the server and not just locally.

You can then define this statement as a Supporting Task in QEMC and set it to run on a schedule or to be triggered by another task.

Regards,

[StefanBackstrand · Accepted solution]

The document preload options in QVS only loads that the document data in unaggregated form into QVS memory. It will not perform any aggregations or rendering of any document objects (charts, listboxes, tables and so on) - that is done by the user or a session. It is when the first tab (or the tab that was active when the QVW document was saved) is "hit" that QVS will actually calculate data and render objects, which will create aggregated data, that in any way will be cached by QVS, either as shared data or private to the user.

In short;

If a document is set to pre-load in QVS, the document will load into memory automatically, but the first user visiting the document will inevitably experience the render of the first (or active) tab. The second user will, most probably, not get that delay, since the data is probably in shared cache (depending on data structure and rules).

If a script executes a client to "hit" the document, the document will load to memory, and the client will trigger the render on the tab, thus saving a "real" user from the render time (but only for that tab and it's objects, if not data is shared between other objects and reused).

The document load and cache build up are in no way bound to any client type or connection methodology.

It should be noted that render time of course is no faster than the document data size, structure and many other factors which is the document designer's responsibility as well as server load and resources.

[#6]

Thanks Stefan. That makes sense.

I guess my hesitation to running a script to 'hit' the document as you and Vlad have suggested is the need for a QV license required for the client. We have expiring passwords (company policy), so using a personal user CAL for the auto script to open the QV doc would require maintenance to update the user password every 60 - 90 days. Not a horrible thing, but just one more thing to keep track of.

I have noticed that once I used the Preload option in the QVS document setting, the application appears to open faster (1 -2 minutes instead of 4 or 5 minutes).

Thanks again for your responses.

[StefanBackstrand]

Yes, your initial user will not have to wait for the document to actually load/extract from disk to memory, and hence will have a shorter first load time.

But, I would not go as far as saying that I recommend to have a script caching the first sheet of a document, since that is not the general idea. The QVS core engine scales extremely well over data, if it is well structured and the document does not have "bloated" sheets or ineffective expressions in charts/tables ("Dashboards" anyone?). Tables can, of course, be especially costly.

If you have a first sheet that takes minutes to load, then the document is badly designed, in my eyes. I put no judgment on your document or data (since I have not seen it), but there are surely ways to get around those loading times.

Ad-hoc calculations is there for a reason, and committing (sometimes user specific) data to memory on a perspective of initial loading times seems to defeat the purpose of this.

I hope that you will find a good way to solve the load times. I would imagine that if one took a look at the data, it would still (as it often is with Qlikview) be amazing how fast the data actually gets aggregated. If not; the data might have to be re-structured or expressions reviewed.

[#8]

Again, I agree with you Stefan about being strategic on document layout (data model, expressions, the landing page) in order to maximize the user experience and decrease the initial 'open' time.

And yes, this application opens to a dashboard -- a beautiful and jam-packed (!) dashboard which summarizes the business performance in many dimensions -- both graphically and numerically. So, it is a trade-off for us -- you have to wait for it to load, but once it loads, welcome to QlikView! The business users are impressed with performance (except the intial load time), but it's us developers who are the most impatient because we are accustomed to QlikView speed.

I could look at staging the dashboard some, so that the most general information calcs first and then more specific detail must be sought after (hide/show, toggle switch, etc.).

Thanks again. (and sorry if I misunderstood your recommendation initially.)

[StefanBackstrand]

Yes, one might get a little spoiled on access times from using modern software too much.

No worries and no need to apologize - just happy to help. I hope that you find a way to handle both load times and a rich dashboard.

And remember, don't use macros. 😉

[#10]

Oh, Macros. I love them (when they work!).

And yes, we are spoiled. QlikView has ruined our patience!

One interesting 'challenge' that we are facing since implementing QlikView (from my perspective) is that anyone access data so quickly now that they have time to think and really dig in and analyze and learn (and make insightful recommendations rather than simply dump data to Excel). Some people don't like to think, though. 😐