Publisher load job blocked due to corrupt QVD - how to recover the job?

[struniger]

Dear all,

Last weekend we had a serious issue on the server as one of our central qvd files got corrupted.

The effect of the corrupt file was that a load job, started via the publisher, basically just froze up and never ran into a timeout nor errored out in any way. The load job's log file simply stops at the reading operation of said file. The QVB process is frozen, system log doesn't show anything, the process just stays there doing nothing. Killing the process was only possible directly on the server via a process manager.

The concerned file unfortunately is the central control mechanism for all our ERP downloads, so we had a bunch of subsequent loads doing exactly the same. This ended up in having all available instances of load processes locked and eventually the entire server got instable.

Testing a load of the corrupt file via the Developer ends in a failed load and reload of the application, the same on the publisher leads to a frozen process.

We are investigating how/why the file was corrupted to avoid that in future,

however I am wondering about the locking behaviour:

So my questions are:

(1) Has anyone of you seen this locking behaviour with a corrupt qvd file?

(2) Is there a way to get the QVB process to exit gracefully in such a case?

Thanks for any hints or tips.

Stefan

Environment:

- Single server environment, virtual, 6 Cores, 64bit, 32GB RAM

- QV 11 SR 2

[#2]

Hi Stefan

It will be hard to tell without looking at the contents of the QVD. QVD's start with XML meta-data, which you can actually open in QlikView (or notepad, if you are willing!)

Are you able to open it this way?

There is no try...catch operation in QV script (as far as I am aware ) but one check you could put in could be to load the QVD as an XML, check the columns in it and and then only load if the result is what you expect. If the XML header itself is also corrupted, then this is more likely to fail  the load script in a regular fashion if there is an issue (as there is not as much data in it as the "proper" data sits below.

EG a process could be

fields:

Load ExpectedField, 1 as expected from presavedlist.xls

outer join (fields)

load FieldName as ExpectedField,FieldName as LoadedField, 1 as Loaded from .... XXX.qvd (xmlsimple....)

temp: load count(*)=0 as FieldCk from fields where expected = 1 and isnull(loaded);

vLoadQVD = peek('fieldCk');

if vloadQVD = -1 then

load .... qvd

end if

Other things you could look at could be the error settings, by setting errormode = 0; will allow the script to ignore any errors. This is obviously risky but will complete the reload task, incase you have any other processes that sit on it.

Erica

[#3]

also.... what is the size of the QVD?

[struniger]

Hi Erica,

The XML header is readable, so the process you have suggested would result in a load (and a crash).

Interestingly enough a qvdnorows() on that file gives me a value (of 136528 rows)...

Btw. filesize of the corrupt file is 633KB, the normal file is around 5MB.

I'll have to play with the errormode a little and see what effects that may have - thanks for the suggestions!

Best regards

Stefan

[marcus_sommer]

Sometimes I have similar issues by manually/automatically loads. I take then simply an older qvd from the shadow copies. But I think that these shadow copies or an overload from system/network-ressources caused the storing-error. It meant its rather a system-problem then a qv-problem unless qv would check the stored qvd after the storing. It could be a possibility to do such checks manually with qv per file-functions.

Perhaps tools like QlikView App: TIQView Tools v2 are also helpful.

- Marcus

[struniger]

Hi Marcus,

Thanks - we have a daily backup, so I went back to get the last 'non-corrupt' file and started from there.

We are now following these threads.

(a) Avoid files from becoming corrupt (theoretically the OS should do that, but hey... so we're now doing a backup before writing the risky file, updating the file, checking for a correct write and restore if not)

(b) Have a backup handy (that's provided by (1) and also by our backend IT)

(c) Try to avoid a freeze of the QVB.exe when encountering a corrupt file --> this is where I'm still stumped...

Best regards

Stefan