topic Re: Publisher Analyzer Tool in Scalability

Publisher Analyzer Tool

hps — Mon, 19 May 2014 09:15:50 GMT

A simple QlikView application to analyze reloads in publisher node(s).

High level points of interest shown with the application:

QVB usage per node
Resource consumption (RAM/CPU) per node
- If performance counters are enabled (Also attached)
Success/failures per day, hour task
Reload durations
- Over time per task
- Step breakdown per reload (Open, reload, save..)
Top 5
- Slowest
- Most common failures
- Most common exceptions

0.60 addition: Minor bug fix and added support for QlikView 12.x

DISCLAIMER: Use as you wish and please provide feedback, but no support is given to the application

Re: Publisher Analyzer Tool

Tue, 17 Jun 2014 19:49:01 GMT

Thanks Hampus! Great utility!

Re: Publisher Analyzer Tool

chandrasarva — Tue, 22 Jul 2014 18:00:22 GMT

its very good utility.

Can we add the processid to the perfmon counters for every QVB spawns. I want to see QVB usage for a particular task/s and i think it can be done if we can pull the QVB rocessid.

Re: Publisher Analyzer Tool

chandrasarva — Tue, 22 Jul 2014 21:47:36 GMT

Also in the server utilization Tab can you tell me for CPU utilization what the negative peaks/utilization indicates

Re: Publisher Analyzer Tool

hps — Wed, 23 Jul 2014 06:52:45 GMT

Hi Chandra,

While it's theoretically possible to do something similar to that it is practically impossible, as each task is not naturally tied to each spawned QVB process. The only way is to add counters for 10K QVB processes and have a formula with an equal amount of processes mapped.

As of 11.2 SR7 these statistics are however printed in the end of each task as a part of "supportability". Check the release notes.

The negative values are due to the fact that the current formula counts CPU usage as; all available minus all IDLE. Sometimes the windows performance counters simply yield the wrong values. I'm sure this could be improved in the application.

You can try changing the CPU utilization (%) formula to this:

=(fabs(max({$<ProcessName={'PROCESS(_TOTAL)'}, ProcessDimension = {'% Processor Time'}>}ServerData))-fabs(avg({$<ProcessName={'PROCESS(IDLE)'}, ProcessDimension = {'% Processor Time'}>}ServerData)))/$(vCores)

Let me know if that helped as I don't have data right now that show negative numbers.

Great that you found it useful.

Regards,

Hampus von Post

Re: Publisher Analyzer Tool

chandrasarva — Wed, 23 Jul 2014 15:55:28 GMT

Hello Hampus,

Thanks for clarifications.

I tried with FABS functions and still no difference. Please let me know if you have any thoughts.

Thanks

Chandra

Re: Publisher Analyzer Tool

chandrasarva — Fri, 25 Jul 2014 17:47:36 GMT

Hampus,

One more question.

why teh max allowed QVB's showing differently for a single set of tasks. I guess teh max allowed QVB's should be consistent across as per the setting in QMC. right.

Thanks

Chandra

Re: Publisher Analyzer Tool

hps — Mon, 28 Jul 2014 08:25:25 GMT

Hi Chandra,

The max allowed is not per individual task, but at some point it was increased from what seems like 12 to 16 parallel QVBs. If you change the chart to per day or similar you can see when the change was done.

And if not changed in the future then that level should be set firm to one value.

Regards,

Hampus von Post

Re: Publisher Analyzer Tool

chandrasarva — Mon, 28 Jul 2014 17:37:57 GMT

Got it. sorry Over looked.

Regards

Chandra

Re: Publisher Analyzer Tool

userid128223 — Tue, 16 Feb 2016 14:00:15 GMT

I don't see attachments for performance monitor. What monitors do i need to configure which will show me CPU & Memory utilization on publisher servers. What is it is cluster of 3 QDS servers.

Re: Publisher Analyzer Tool

hps — Wed, 17 Feb 2016 07:57:31 GMT

Hi,

The performance counters are not included in this .qvw.

But they are with the QVscriptGenTool.

QV Scalability Tools (QV10,QV11,QV12)

I will try to update the publisher analyzer to include this file in any updates I make.

Regards,

Hampus von Post

Re: Publisher Analyzer Tool

psankepalli — Wed, 20 Apr 2016 00:31:01 GMT

Superb tool.. great work..

Re: Publisher Analyzer Tool

Thu, 23 Jun 2016 11:16:25 GMT

Hello !!

i'm facing an issue with the tool he's detecting the date on another format :s

the month is detected as a day and vice versa if you can help me on this as soon as possible plz

thanks

Re: Publisher Analyzer Tool

rahulsud007 — Thu, 07 Jul 2016 04:22:56 GMT

Hi Hampus, this is a great tool. Thanks for the upload.

I was trying to use this application, but when I reload I was getting the following error,

Attached is the error which we r getting. Kindly suggest.

Thank you !

Re: Publisher Analyzer Tool

hps — Thu, 07 Jul 2016 08:15:15 GMT

Hi Rahul S,

Late reply but this error occurs when the analyzer is not properly able to extract the timestamp from the logs.

I gather the timestamp in two places.

First on the Root Load tab where I try to extract the times. Here I've added US timestamps as well as your local QlikView default as set in "SET TimestampFormat='YYYY-MM-DD hh:mm:ss[.fff]';"

The second places where I suspect that your script fails is in the PublisherLogs where the ALT query is a bit larger as seen below:

timestamp(floor(alt(timestamp#(@1,'MM/DD/YYYY hh:mm:ss'),timestamp#(@1,'MM/DD/YYYY hh:mm:ss TT'),timestamp#(@1,'MM/DD/YYYY hh:mm:ss.fff'),timestamp#(subfield(@1,'T',1)&left(subfield(@1,'T',-1),6),'YYYYMMDDhhmmss'), @1), '.0000001')) as DateTime,

If neither of those formats matches your timestamp format or the default set by header then you will get this error.

So the question is now. What format is the first column in the Rootlog and a sample publisher log?

And what is your default SET setting otherwise?

Then you should be able to modify the script to get going.

Regards,

Hampus von Post

Re: Publisher Analyzer Tool

hps — Thu, 07 Jul 2016 08:15:57 GMT

Also I think there was a log format change between QlikView 11.2 and QlikView 12.

Which version are you using?

Re: Publisher Analyzer Tool

rahulsud007 — Thu, 07 Jul 2016 11:39:09 GMT

Hi Hampus, you were right date format needed to be change. I have changed it, its working fyn now. Thank you.

Is there anyway I would be able to find the reload start time of triggered qvw application?

Suppose I have two applications, QV-Gen1 and QV- Gen2. I would like to have reload start time of QV-Gen2.

Re: Publisher Analyzer Tool

hps — Thu, 07 Jul 2016 13:05:32 GMT

Hi,

The easiest way is to mark the task/document you want to view and look at the "TaskStart" field, which would be what you are looking for.

Do you mind posting a sample of your logs timestamp format so I could append that to the document for future reference?

Thanks,

Hampus von Post

Re: Publisher Analyzer Tool

hian_cintra — Thu, 07 Dec 2017 14:00:23 GMT

Hi Hampus, how you doing?

By first, thank you for this tool, very very useful for us, let me ask you:

Is there any known issue on Qlikview 12? When i changed my environment to the version 12, the Publisher Analysys does not show the document list anymore.

Can you tell me where os what to analyse on the script?

Re: Publisher Analyzer Tool

hps — Fri, 08 Dec 2017 10:12:34 GMT

Hi,

I've uploaded a new version.

Try it out as it should work for QV 12.

Regards,

Hampus von Post