topic Re: Qlikview Log File Analyzer in QlikView

Qlikview log analyser

jansen28 — Wed, 21 Jan 2015 15:03:33 GMT

Hi Forum folks!

The problem we are facing in log file is that, if Qlikview document is having large amount of load script, it is currently difficult to find how much time each piece of the load script is taking time in document logs.

To resolve this we are plannig to implement a solution called "Document log analyser" which will analyse a set of qlikview document logs and capture the execution duration of each script statement in a reload.

Is there any keyword search in qlikivew where I could search for all keywords "loads" and "fields found" and extract that particular line and store it in a file. then compare 1st row and 2nd row and store the difference between separately....

Is this possible to do in qlikview? if yes, How to achieve it?

Any help greatly appreciated.

Re: Qlikview Log File Analyzer

marcus_sommer — Wed, 21 Jan 2015 19:39:13 GMT

I have heard that somebody had already built such log-file analyzer but I can't remember who. But I think that approaches like these within in link fit better in a long term:

http://community.qlik.com/docs/DOC-7177#comment-20906

- Marcus

Re: Qlikview Log File Analyzer

rwunderlich — Wed, 21 Jan 2015 19:43:33 GMT

A Script Log Analysis tool is already written and freely available at:

Qlikview Cookbook: Script Log Analyzer http://qlikviewcookbook.com/recipes/download-info/script-log-analyzer/

-Rob

http://masterssummit.com

http://robwunderlich.com

Re: Qlikview Log File Analyzer

darrellbutler — Fri, 23 Jan 2015 10:00:14 GMT

Rob,

Many thanks for your power tools, they're indispensable. I have one small issue with script log analyzer, it keeps falling down at line 1988 when trying to left Join onto LogSummary with the reason table not found. Any ideas what might be causing this ?

Kind regards

Darrell

Re: Qlikview Log File Analyzer

Fri, 23 Jan 2015 10:21:01 GMT

Hi Janani,

you might also find this

http://community.qlik.com/docs/DOC-6187

helpful for more customised audit logging to help you with sql load times, qvd store times, row counts etc etc

hope that helps

Joe

Re: Qlikview Log File Analyzer

Peter_Cammaert — Fri, 23 Jan 2015 11:06:25 GMT

The ScriptLogAnalyzer reads logs with file mask *.qvw.log. Those are the real document logs generated by QV Desktop. It won't read QDS logs à la DocumentLog.txt, except if you rename them. And I don't know (yet) what happens with the wide timestamps in those logs.

Re: Qlikview Log File Analyzer

darrellbutler — Fri, 23 Jan 2015 11:20:51 GMT

Peter,

Thanks for your reply, am I correct in assuming that the logs that are fed into the script analyzer are the logs saved to:

C:\ProgramData\Qliktech\DistributionService\1\Log and have generate logfile turned on at the document properties level ?

Thanks

Re: Qlikview Log File Analyzer

Peter_Cammaert — Fri, 23 Jan 2015 11:33:04 GMT

QDS = QlikView Distribution Service. It won't read QDS logs for two reasons, I think:

Those logs have the wrong filenames (not matching *.qvw.log)
Their layout is different .QDS logs have wide (double) timestamp areas, while the Desktop logs don't.

Use the ScripLogAnalyzer only with actual document logs. In a server environment and with "Generate Log File" enabled, you can find them in the same directory as the published documents.

Rob Wunderlich maybe you can include an extra test in your wonderful tool: if table LogSummary is empty, skip the remainder of the script.

Re: Qlikview Log File Analyzer

darrellbutler — Fri, 23 Jan 2015 11:43:37 GMT

Thanks for your help Peter, I understand which logs I should be using. However, it still seems to be falling down at the same point !

Re: Qlikview Log File Analyzer

Peter_Cammaert — Fri, 23 Jan 2015 11:49:53 GMT

Well, I got it to work ok as soon as I pointed the ScriptLogAnalyser to a set of Desktop-generated logs. Just a moment, I'll try with logs in the server.

Re: Qlikview Log File Analyzer

Peter_Cammaert — Fri, 23 Jan 2015 11:54:30 GMT

It still works with logs generated in the QVS document root.

Can you post a screenshot of the directory with the logs? Please include the full file name (including extensions).

Re: Qlikview Log File Analyzer

darrellbutler — Fri, 23 Jan 2015 13:00:34 GMT

Hi Peter,

Here goes

Re: Qlikview Log File Analyzer

rwunderlich — Fri, 23 Jan 2015 19:33:56 GMT

Thanks for sorting this out Peter! I've posted an updated V1.3 at the above link that fixes the script error when no logs are found.

It still won't load QDS logs, because as you pointed the format is different. It could be made to do that, I may include that in a future update.

-Rob

Re: Qlikview Log File Analyzer

jansen28 — Thu, 05 Feb 2015 08:24:00 GMT

All, Thanks for you valuable reply:)

My special thanks to Rob Wunderlich , I would like to call you as "Qlikview guru". It might take ages for me just to understand your code. You did "impossible" in qlikview as "possible"

Re: Qlikview Log File Analyzer

Fri, 13 Feb 2015 09:50:09 GMT

Hello all,

what is the reload time for this application?

Do you know what is the issue?

Mine is still loading, and it already passed the 40m minute mark.

See the screenshot.

Best regards

Re: Qlikview Log File Analyzer

darrellbutler — Fri, 13 Feb 2015 10:15:09 GMT

Nemanja,

I guess this depends on the number of log files you're loading into the app, I tried it with several and it did take some time. There's a fair bit of meaty code in this app !

I use this on an individual app basis to root cause performance issues, I don't load several logs from several applications - depends what issue you're trying to resolve I suppose.

What's happening on your QV server ? is there plenty of spare capacity ?

Regards

Darrell

Re: Qlikview Log File Analyzer

Fri, 13 Feb 2015 10:17:02 GMT

Hi there,

so the issue is with the amount of data that is being loaded.

Thanks for the info.

Best regards,

Nemanja

Re: Qlikview Log File Analyzer

rwunderlich — Mon, 16 Feb 2015 17:45:20 GMT

The screenshot you posted shows only one second of elapsed time. Is that the way the screen stays?

My best guess is that you have not given a valid path in the config and that it is scanning the entire disk. Is that possible?

-Rob

Re: Qlikview Log File Analyzer

Thu, 19 Feb 2015 09:35:26 GMT

Hello Rod,

the screen stayed like that for a long time.

But now I figured out the issue, and everything is working as intended.

Best regards

Re: Qlikview log analyser

rwunderlich — Thu, 19 Feb 2015 16:04:54 GMT

Can you tell us what the issue was?

Rob