Preloading Documents

[rmueller]

So, we have two servers, I'll call them Inside and Outside.

Inside does document loading and distribution to Outside.

Outside runs our Extranet services, exposing documents published by Inside, it currently does NOT distribute documents.

I want to 'Preload' documents on Outside that are reloaded/distributed from Inside, in which QMC (Inside or Outside) should I setup the document control to make that happen?

ron.

[Miguel_Angel_Baeyens]

If you have two different QMCs, the one that governs the Outside, then. In QMC > Documents > User Documents you can expand and select which document you want to have preloaded clicking on the document > Server > Performance.

[rmueller]

Yep, knew that. 

In fact, we can set 'preload' on documents found on Outside server from either QMC (both QMC's show the user documents, Inside sees them via a share).  However, setting the flag doesn't 'preload' the documents.  Publishing a document from Inside to Outside doesn't preload the document either.  So, setting the flag enables that document to be preloaded, but doesn't actually pre-load it on the Outside server.

For instance, if i restart QVS, memory starts very low.  I publish a large document (with preload checked or 'ON') from Inside to Outside, QVS memory stays the same after the publishing; clearly the document did not preload.  I then open the same document, QVS memory climbs.

The documents only load into memory and stay there once a user opens one of them.  Evidenced by the qlikviewserver memory climbing after each document is opened, and it stays there when they are closed, but I have no way to actually confirm this is due to document 'preloading'.

This is not the behavior I would have expected.  I would have expected that the document actually were 'preloaded' before a user touched them.

Is this proper behavior?

ron

[Miguel_Angel_Baeyens]

Your expectations are correct and does is not the intended behavior. I can see without any interaction how a couple of minutes after the server is started how memory starts to grow.

Check with Qlik Support in case the QMC reflects the flag as set, but for some reason it is not so, and the .ini or .xml file which should be storing that is not doing it.

However, it is not clear for me how the QMC is governing two different servers. Are they clustered but somehow one is exposed and the other is not or are the completely separated environments, although they can be sharing folders?

As for the memory, it will stay used as long as the service is running. If the document times out or there is a new copy, it will even keep memory used as long as it can be cached (e.g.: data model remains the same but there are more lines after a reload). This is intended and expected.

[rmueller · Accepted solution]

That's what I thought....not exactly how it should work, but it is how it does work.

The Inside is a fully licensed environment, server, publisher...etc.

The Outside is an 'Extranet' server license; basically everything but publisher (same code both servers, just a licensing difference).

I think its because the Outside never actually 'publishes' anything (can't publish either, it doesn't have a license to publish).

The Inside does the publishing to a shared folder on Outside, this doesn't trigger Outside to 'preload' the document.  Basically, Outside QVS doesn't know the document exists (QMC sees it, and it can be seen in Access point), until the document is opened via the Access point on Outside.

The Inside publishes to shared folders on the Outside, both QMC's mount the shared folders; therefore both of them 'see' the documents and can change properties associated with them...ie. 'Preload'.

Let me also add...since each time Inside 'publishes' to Outside a 'new' document file is created...this clears the 'preload' flag each time.  So, the right thing to do is set the 'preload' on the publishing Task, not the actual document in Documents/User Documents.  That way each time the flag is preserved (actually reset each time).

Ron

[Miguel_Angel_Baeyens]

The preload does not work, at least not as far as my knowledge and experience goes, only when a document is published, but when the check is applied for the document. It might not be immediate, but take a few minutes, but if you click on Preload I expect to see, sooner rather than later, an increase in memory usage corresponding to this document being loaded.

Unless, of course, the document is already in memory.

Check with Qlik Support in case my understanding is not correct or is a bug (or a limitation due to the licensing).

[rmueller]

Yep, time to call support...

I did this experiment.  Set the preload in the publisher task on Inside, so when documents are 'published' to Outside the flag is set each time on the document.  That works perfectly.

I shutdown QVS, deleted all the log files so everything is fresh.

Restarted QVS

I can see QVS has found the documents that should be preloaded, however it fails to actually load them!

20181023T095851.000-0500	20181023T095852.000-0500	4	700	Information	DOC loading: Beginning load of document Q:\COGITOPUBLISHEDDOCUMENTS\COGITO ANALYTICS.QVW.
20181023T095851.000-0500	20181023T095852.000-0500	4	700	Information	DOC loading: Beginning load of document Q:\COGITOPUBLISHEDDOCUMENTS\MANIFEST ANALYTICS.QVW.
20181023T095851.000-0500	20181023T095852.000-0500	4	700	Information	DOC loading: Beginning load of document Q:\COGITOPUBLISHEDDOCUMENTS\WORK FORCE EFFICIENCY MKTG CUSTOM.QVW.
20181023T095851.000-0500	20181023T095852.000-0500	4	700	Information	DOC loading: Beginning load of document Q:\COGITOPUBLISHEDDOCUMENTS\WORK FORCE EFFICIENCY MKTG EXPLORE.QVW.
20181023T095851.000-0500	20181023T095853.000-0500	4	700	Information	DOC loading: Beginning load of document Q:\COGITOPUBLISHEDDOCUMENTS\WORK FORCE EFFICIENCY MKTG INTRO.QVW.
20181023T095851.000-0500	20181023T095853.000-0500	1	300	Error	Server: DoLoad (TSmarterPtrBase<CServerDocumentEntry>): Internal error
20181023T095851.000-0500	20181023T095853.000-0500	1	300	Error	Server: DoLoad (TSmarterPtrBase<CServerDocumentEntry>): Internal error
20181023T095851.000-0500	20181023T095853.000-0500	2	500	Warning	Document Load: The document Q:\COGITOPUBLISHEDDOCUMENTS\Work Force Efficiency Mktg Custom.qvw failed to load.
20181023T095851.000-0500	20181023T095853.000-0500	1	300	Error	DOC loading: Exception while loading document Q:\COGITOPUBLISHEDDOCUMENTS\WORK FORCE EFFICIENCY MKTG CUSTOM.QVW.
20181023T095851.000-0500	20181023T095853.000-0500	2	500	Warning	Document Load: The document Q:\COGITOPUBLISHEDDOCUMENTS\Work Force Efficiency Mktg Explore.qvw failed to load.

[rmueller]

Another note...since both Inside and Outside can see the published documents in the share on Outside...here's what happens.

I set the 'preload' in the published task - this guarantees the document 'preload' is set each time the document is published.

I shutdown QVS on Inside, deleted all the log files just make sure everything is clean to start.

I start QVS on Inside

QVS on Inside begins to load all the documents in the published share that have the 'preload' flag set.  Each document successfully loads on Inside.  That's great, but I don't want these documents loaded on Inside, just Outside.  The Outside server doesn't show up on the dropdown when I click custom...

Bottom line, 'preloading' kinda works, but not for an Inside/Outside configuration.

[Miguel_Angel_Baeyens]

Good news, somehow.

Anyway, since you need a Publisher to distribute to a QES, I would still check with Qlik Support whether there is any way to have this or if there is a known limitation (license or otherwise) that does not allow preloading (and likely, other settings as well) to be set for the QES.