Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Launch different tasks referred to the same qvw at the same time

I use a qvw document to make ETL in order to create QVDs,

(depending on input variable)

is it possible to launch at the same time different tasks referred to the same document?

(every task launches the same qvw but with a different variable/value input - tab Reload - Script parameter)

Every task make a instance of the qvw document?

Thanks in advance

Mary

12 Replies
Miguel_Angel_Baeyens

Hi,

Do you have QlikView Server and Publisher? If so, you can create a task to loop for all values in a field and create as many QVW as values in the field are.

If not, then you will have to somehow (i. e.: using a Windows BAT file) run all possible values and pass one by one to the qv.exe file. Anyway, if this kind of reload is a must, I'd rather check with your QlikView sales rep or partner manager so he or she can give you some advice on the best software license to get Publisher working.

Note that there is a limit in the number of concurrent reloads you may have, depending on your hardware in order to prevent memory pagination, usually the number of cores the computer you are running QlikView has minus one.

Hope that helps.

Miguel

pat_agen
Specialist
Specialist

Hi Miguel,

getting to grips with QMC and using the reload functionality for the first time.

In a similar vein to Mary's question can you confirm that from QMC on v11(SR1) I can only have one refresh enabled per physical qvw. It looks that way but I just want to make sure I haven't overlooked anything.

Thanks for your comments.

Not applicable
Author

Hi Miguel

thanks for your quick response!

Yes, I have a server + publisher, QlikView Version:10.00.9086.7

but I need to reload on demand the qvw for a single value that I receive in input (edx).

and it' possible that I receive different call for the same qvw but with different value.

Please see my previous post:

http://community.qlik.com/message/263444#263444

Mary

Not applicable
Author

Hi, Migual

You wrote:

"you can create a task to loop for all values in a field and create as many QVW as values in the field are"

The issue is : with the same qvw file in parallel to make many (more than 9 qvb.exe processes at the same time)different qvd files using parameter.

And what is your opinion about that :

There are some registry settings you can change to increase the number of com communication threads allowed per machine.

Change the registry key and restart the server


Change the registry setting:

            HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\SubSystems\Windows

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows

               SharedSection=1024,3072,512 Windows=On SubSystemType=Windows

               ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3

               ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off

               MaxRequestThreads=16

(Default is 1024,3072,512 in 32bit or 1024,3072,768 in x64)


To:

Change the GDI and User handle max count in the registry to SharedSection=1024,20480,2048

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows

               SharedSection=1024,20480,2048 Windows=On SubSystemType=Windows

               ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3

               ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off

               MaxRequestThreads=16

Best regards,

Hugo

Miguel_Angel_Baeyens

Hi Hugo,

Not sure if I got your question right but, in order to have a good performance and avoid corruption of files and degradation in the use of CPU and RAM, having too many distribution engines should always equal to the number of cores minus 1.

I know the registry settings, that are OK if you have a large number of cores (say 32) and your operating system does not recognize them somehow.

If you have the same QVW being looped and reduced, you better have several different fields to loop by, otherwise you will be launching the same task 9 times instead of launching 9 different tasks with different values in the field to reduce.

Hope that makes sense.

Miguel

Not applicable
Author

Hi

and what about docuntation from SR2:

"

Horizontal Scalability


Horizontal scaling of hardware provides the ability to increase the resources of the QlikView deployment. By adding additional hardware servers, the workload of QlikView Publisher can be increased. The clustered Publisher servers can then be configured to load balance the QlikView tasks.


For example, on a certain hardware server, QlikView Publisher can process 100 concurrent tasks.

"

does it mean 100 core per machine ?

Best regards

Hugo

Miguel_Angel_Baeyens

Hi Hugo,

That speaks about a clustered environment, where more than one computer is dedicated to distribution tasks (Publisher). How long do the tasks take to reload? How good is the network connection? How fast is the speed writing to the hard disk? How big are the reduced or distributed files? How many people are authorized in each document and how are they authorized? Do they use the same sources (QVD, XLS, Databases) at the same time? Version 11 or version 10?

Anyway, 100 concurrent tasks in the same hardware seems pretty much to me, and obviously a huge computer with top notch hardware, fastest CPU processors, highest speed memory bus and lots of cores...

There are lots of questions to answer to get the most accurate recommendation, because there are no two equal QlikView deployments and they will vary widely from customer to customer and from document to document.

My experience is that as a general rule, more than 8 concurrent tasks in a single computer starts taking a lot of time.

Hope that makes sense.

Miguel

Not applicable
Author

Hi,

Does anybody you had heard has changed registry to "SharedSection=1024,20480,2048" and sucessfully reloads more than 8-9 qvw concurrently ?

Hugo

Bill_Britt
Former Employee
Former Employee

You really don't need to set your QVB engines count to more that the processors you have. The setting that Migeul increases a limitation that Microsoft puts on the number of comm objects that can really one at the same time. Yes, this setting works, but at times you have to play with it to get it working right.

Here are the Microsoft articles.

http://blogs.msdn.com/ntdebugging/archive/2007/07/05/desktop-heap-part-2.aspx

http://blogs.msdn.com/ntdebugging/archive/2007/01/04/desktop-heap-overview.aspx

NOTE: Make a backup of your Registry before you make any changes.

Bill


Bill - Principal Technical Support Engineer at Qlik
To help users find verified answers, please don't forget to use the "Accept as Solution" button on any posts that helped you resolve your problem or question.