QlikView Server: overlapping time slices

[Anonymous]

Hi community,

I have a question concerning QlikView Server scheduling. Basically I want to store more data in qvd files...

(I don't use QlikView Publisher)

An imaginary example:

I have 5 slowly changing dimension tables stored in 5 separate qvd files.

For each of the 5 tables I have a separate qvw which updates the concerning qvd.

Each of the 5 qvw is reloaded with different reload times, for example:

- qvw1: each 5 minutes

- qvw2: each 43 minutes

- qvw3: each 9 minutes

- qvw4: once each day

- qvw5: each 2 minutes

Then a have one "main-qvw" which loads data from qvw1, ..., qvw5 will be reloaded each 29 minutes...

My question is:

What happens, if the update process of one of the 5 qvds happens exactly in the same moment when the "main-qvw" reads data out of the concerning qvd?

Will it lead to a "crash" or is QlikView server capable handling this issue.

(I ask this question because it is hard to simulate that reading and writing hapen exactly in moment... I can't just stop the QlikView Server and create a test scenario)

Regards Robin

[marcus_sommer]

My experience is that it could respectively will lead to conflicts if there is a file-lock on the qvd. Therefore I would try to avoid this situation. If you could make slightly changes to the frequencies and when they exactly start and their reloading-times (to shorten will be rather not possible but with sleep-statements you could delay them). For example:

qvw5 runs to each even minute

qvw1 runs in minute 1, 6, 11 ...

qvw3 runs in minute 3, 13, 23 ...

qvw4 runs in minute 59

bei qvw2 I'm not sure how to fit it in this logic - especially because the reloading-and also the storing-times needs to considered.

Another approach could be to implement a kind of a reload-check. I mean that each qvw creates another qvd as a reload-parameter, for example:

qvw5

ReloadParameter:

load 'in use' as ReloadParameter autogenerate 1;

store ReloadParameter into ReloadParameterQVW5.qvd (qvd);

... the origin load-stuff

ReloadParameter:

load 'closed' as ReloadParameter autogenerate 1;

store ReloadParameter into ReloadParameterQVW5.qvd (qvd);

and your main-qvw checks these parameters before accessing the file and does wait a while if its in use. And of course the same method could be applied revert if the main-qvw is in work and the other qvw's want to update the qvd's.

I could imagine that this logic could be outsourced in a sub-routine which just update a further variable as a control for the origin loadings so that script won't be clutterred too much.

- Marcus

[Anonymous]

... I suspected something like that

I think the ideas with "in use " flags is a good one!

Optimizing the timing makes no sense for me I think, because I have multiple reports running on the server and I set that a maximum of 3 reports are allowed to load concurrently...

So the rest of the reports is in "waiting queue" status an so the real starting time is not exactly predictable.

Perhaps it is possible to check, if the reading/writing of a qvw was successful and if not just try it again.

--> But this could lead to endless loops in certain situations I think...

[marcus_sommer]

I would restrict the number of possible loops (max. number and time-span to wait could be parameter from a sub-routine) to a certain max. value on which the routine ends itself.

Further I could imagine that this approach could be replaced and/or extended to routines which copy and delete the qvd's per EXECUTE batches and the other qvw's check if the expected qvd exists and is newer than the last one and loads from there (I assume that those qvd's are rather small). Of course this adds further complexity but it could be kept in a sub-routine and/or include-variables to keep it practically.

Another thought is the use of the ERRORMODE. I don't use it in productive and I have never checked if it could handle problems with file-locks by reading/writing qvd-files whereby I remember postings in which people have tried it - also with a small loop and waiting and again. I think you will need to check it on your system.

- Marcus

[Anonymous]

thanks

[Anonymous]

I experimented with ErrorMode=0 and provocated an exclusive lock on a qvd with the help of a tool.

1) trying to read a locked qvd

--> means I could create a loop together with sleep command until I get no error... hopefully

--> could try it 10 times and then let the script continue --> will procate an error when setting back ErrorMode <> 0

2) trying to write a locked qvd

The app crashes even with ErrorMode = 0 --> so using a separate lock file is mandatory.

I'm still not sure, if reading a qvd locks a file exclusively.

Does anybode have further information?

(QV11.2 SR15)

[Anonymous · Accepted solution]

found out that ErrorMode when writing a qvd works with QlikView 12:

So there aren't any separate lock status files needed anymore when using QV12.

--> see the comment by  rwunderlich‌