Why reload failed when the RAM usage is not super high?

[Huiying]

Hi

I have a monster qv document. Normally the reload fails due to resource issue. I got the typical error 'Error: Document open call failed. The document might require username and password.' while QVB is loading the document.

Today it failed strangely. Our QDS max RAM is 160G. At 1:42am, the server peaked to the maximum, but it survived.

At 2:30, it failed for a much smaller spike.

How to understand this? Is the resource the real issue?

(23.7.2019 1.38.22) Information: Slow down logging. Log every <900> seconds.

(23.7.2019 1.38.22) Information: Reloading

(23.7.2019 1.48.39) Information: The Source Document reload complete. DocumentPath=\\files\QVS4\SourceDocuments\D\D.qvw

(23.7.2019 1.48.39) Information: Memory Allocation Delta for this file=60512.33 Mb. Available Physical Memory Before Reload=93946.82 Mb. Available Physical Memory After Reload=33257.93 Mb.

(23.7.2019 1.48.40) Information: The Source Document was reloaded successfully.

(23.7.2019 1.48.40) Information: QlikView->Settings->Document Peferences->Generate Logfile in document is set. Enabling Log Copier.

(23.7.2019 1.48.40) Information: Looking for the document log "\\files\QVS4\SourceDocuments\D\D.qvw" to become available.

(23.7.2019 1.48.40) Information: Found a document log. FileName=\\files\QVS4\SourceDocuments\D\D.qvw.log

(23.7.2019 1.48.40) Information: Writing documentLog to \\files\QVS4\QVLogs\QDS\1\Log\20190722\223055 - D_FullLoad\DocumentLog.txt

(23.7.2019 1.48.41) Information: Reload finished successfully

(23.7.2019 1.48.41) Information: Start saving document

(23.7.2019 2.22.04) Information: Check in document: \\files\QVS4\SourceDocuments\D\D.qvw

(23.7.2019 2.22.04) Information: Closing the document.

(23.7.2019 2.22.27) Information: Closed the QlikView Engine successfully. ProcessID=614468

(23.7.2019 2.22.27) Information: Start document distribution.

(23.7.2019 2.22.27) Information: Opening source document

(23.7.2019 2.22.27) Information: Opening document: "D.qvw"

(23.7.2019 2.22.27) Information: Allocating new QlikView Engine. Current usage count=0 of 8 (of type non-reader).

(23.7.2019 2.22.27) Information: Max retries:5

(23.7.2019 2.22.27) Information: Attempt:01

(23.7.2019 2.22.28) Information: Opened the QlikView Engine successfully. ProcessID=378744

(23.7.2019 2.22.28) Information: Allocated QlikView Engine successfully. Current usage count=1 of 8 (of type non-reader). Ticket number=276.

(23.7.2019 2.22.28) Information: Loading document "\\files\QVS4\SourceDocuments\D\D.qvw" (9493.00 Mb)

(23.7.2019 2.22.29) Information: Loading. LoadTime=00:00:01.0057118

(23.7.2019 2.22.31) Information: Loading. LoadTime=00:00:03.0364084

(23.7.2019 2.22.35) Information: Loading. LoadTime=00:00:07.0786430

(23.7.2019 2.22.43) Information: Loading. LoadTime=00:00:15.1565142

(23.7.2019 2.22.59) Information: Loading. LoadTime=00:00:31.2727980

(23.7.2019 2.23.31) Information: Loading. LoadTime=00:01:03.2881789

(23.7.2019 2.24.32) Information: Loading. LoadTime=00:02:03.4865478

(23.7.2019 2.25.32) Information: Loading. LoadTime=00:03:03.9699050

(23.7.2019 2.26.32) Information: Loading. LoadTime=00:04:04.2797674

(23.7.2019 2.27.32) Information: Loading. LoadTime=00:05:04.4291582

(23.7.2019 2.28.33) Information: Loading. LoadTime=00:06:04.8917904

(23.7.2019 2.29.33) Information: Loading. LoadTime=00:07:05.2782997

(23.7.2019 2.30.33) Information: Loading. LoadTime=00:08:05.3103588

(23.7.2019 2.30.55) Error: Document open call failed. The document might require username and password.

(23.7.2019 2.30.55) Information: Attempted to load the document with data.

(23.7.2019 2.30.55) Error: The document failed to open.

(23.7.2019 2.30.56) Information: Closed the QlikView Engine successfully. ProcessID=378744

(23.7.2019 2.30.56) Error: Document could not be opened

(23.7.2019 2.30.56) Information: Closed the QlikView Engine successfully. ProcessID=378744

(23.7.2019 2.30.56) Information: Document distribution done.

(23.7.2019 2.30.57) Error: The task "D_FullLoad" failed. Exception: || QDSMain.Exceptions.TaskFailedException: Task execution failed with errors to follow. ---> QDSMain.Exceptions.FailedDocumentCheckoutException: Failed to open document: \\files\QVS4\SourceDocuments\D\D.qvw || at QDSMain.DistributeTask.OpenDocument(IExecutingTaskResult executingTaskResult, String fileName) || at QDSMain.DistributeTask.Distribution(String fileName, DistributionRequest distributionRequest, IExecutingTaskResult executingTaskResult, String repeatVariableName, String currentRepeatVariableValue) || at QDSMain.DistributeTask.PerformExecute(IExecutingTaskResult executingTaskResult) || --- End of inner exception stack trace --- || at QDSMain.DistributeTask.PerformExecute(IExecutingTaskResult executingTaskResult) || at QDSMain.Task.AbstractTask.TaskExecution(CurrentExecutionArgs args)

(23.7.2019 2.30.57) Information: Task Execute Duration=04:00:00.6288274

(23.7.2019 2.30.57) Information: TaskResult.status=Finished

(23.7.2019 2.30.58) Information: Notifying all triggers of new state: "FinishedWithErrors"...

(23.7.2019 2.30.58) Information: Notifying all triggers of new state: "FinishedWithErrors" - Completed

[marcus_sommer · Accepted solution]

Maybe your server didn't really survived even if it not crashed directly. Quite a lot of measurements aren't directly executed from Qlik else through the various features and services of the OS, for example all handlings to the file-system. If now the RAM hit the limits of your environment some parts of those features might be swapped to the hd and may fail through this in any way. It could cause very strange issues and often it's not possible to find the real reason at least not without an expensive process-monitoring. Therefore you need to find ways to ensure that your system has always enough RAM available.

Beside the usual suggestion to increase your hardware-resources it might be also helpful to ensure that not multiple tasks run in parallel and of course if your applications couldn't be more optimized.

- Marcus

[Sonja_Bauernfeind]

Couldn't really have said it any better than @marcus_sommer 

Resource starvation is likely to cause larger issues in the system, and there might be more information to be found in the Windows System or Application log if a root cause analysis is necessary. 

/Sonja

[Huiying · Accepted solution]

thank you for the response.

Now we found the root cause: due to human error or something, we had 3m extral rows. That was difficult to troublshoot as no error in document log.

[Albert_Candelario]

Thanks for sharing the root cause with us!