Qlik Community

Ask a Question

QlikView Management

Discussion Board for collaboration on QlikView Management.

Announcements
Welcome to our newly redesigned Qlik Community! Read our blog to learn about all the new updates: READ BLOG and REPORTED ISSUES
cancel
Showing results for 
Search instead for 
Did you mean: 
Creator
Creator

Known problems with parallel loads in QlikView Server?

Hello everyone!

For a customer we are administrating a QlikView Server 11.2 SR12 Small Business Edition. Currently, we have the task chain setup for all tasks to run in sequence. We have set it up that way, because at some point we were experiencing problems with the tasks (while still loading in parallel) and our suspicion was that concurrent QVD access (read access only) was the cause of it.

Now we are eager to speed up the nightly reload and use the servers parallel load abilities again. Did anyone ever have issues due to apps READING from the same QVD file at the same time? Would there be any reason for this to be problematic?

Thanks!

Sandro

1 Solution

Accepted Solutions

Hmm. Should we add some nuances to Ian's comment (case 1) in QVD read and write at the same time and consequences?

Also, Marcel could add a similar technique to failing LOADs as he did in a smart way to failing STOREs in Avoid General Script Error in STORE command

View solution in original post

8 Replies
Support
Support

HI,

This will be an issue, because of the QVD being locked.

Bill

Bill - Designated Support Engineer at Qlik
To help users find verified answers, please don't forget to use the "Accept as Solution" button on any posts that helped you resolve your problem or question.

IMHO a locked file does not block other processes from opening it for Reading (on condition that it's not an Exclusive Lock) It's the Writing combined with attempts to Read the same file that causes problems.

Just did multiple tests with 4 reloads running on the four cores of my laptop (along the way locking up Windows as the reloads kicked the cpu into 100% load) all reading the same 3Gig QVD with lots of WHERE conditions. All went fine, even the one where the QDS mistakenly started four reloads on a single core (no queueing).

Bill, would you advise never to open a single QVD for reading in two or more reload processes at the same time?

Peter

Support
Support

Hi Peter,

This is an know issue when using publisher to reload task. So, if they try and task start to fail it is because of the locking issue. I think the issues comes in on the initial opening of the file and task stepping on each other.

Bill

Bill - Designated Support Engineer at Qlik
To help users find verified answers, please don't forget to use the "Accept as Solution" button on any posts that helped you resolve your problem or question.

So it is a timing issue? If tasks all try to open a single QVD at the exact same time, they may get errors? A simple retry (optionally with a delay of a few seconds) may fix this?

Peter

Support
Support

Hi Peter,

You are correct and I should have posted that.

Bill

Bill - Designated Support Engineer at Qlik
To help users find verified answers, please don't forget to use the "Accept as Solution" button on any posts that helped you resolve your problem or question.

Hmm. Should we add some nuances to Ian's comment (case 1) in QVD read and write at the same time and consequences?

Also, Marcel could add a similar technique to failing LOADs as he did in a smart way to failing STOREs in Avoid General Script Error in STORE command

View solution in original post

Creator
Creator

Alright guys, thank you for your inputs.

In general I would assume, that it works like Ian wrote in the other thread or as Peter wrote about from his tests (thanks for that). What Bill says is what I was afraid can happen.

Now:

Bill explicitly mentions the Publisher. Just to clarify: does the same hold true for the QlikView Server?

If even two reading tasks could step on each others toes with bad timing, what's the conclusion?

Is the parallel load feature in QlikView Server useless?

Does that add a lot of complexity because you have to make sure that the tasks, which load in parallel do not access the same files at the same time?

Or would it be necessary to wrap all read statements into a loop as described in the "Avoid General Script Error" thread?

How do you guys handle this?

Thank you, very productive this thread 🙂

Sandro

Some quick feedback on one of your questions: you do have a point there. Bill explicitly mentioned the publisher while I did my tests on a QlikView server without publisher.

Now that I think of it: even in the high-volume publisher-based environments that I visit/manage, I've never encountered QVD errors caused by simultaneous Reads. On the other hand, STORE errors do happen occasionally. Of course, that doesn't prove that simultaneous reads are absolutely safe, which is your main concern.

BTW the reload task setup for a QVS without publisher and a QVS with publisher isn't that different. Both environments use a QDS-QVB combo to track schedules, trigger tasks and reload documents. AFAIK the one doing the reloads (and the QVD reading), QVB.exe is exactly the same whether you use a full publisher or a limited one...

Peter