Skip to main content
Announcements
Marching toward a simplified navigation! READ ON

LogAnalysis App: The Qlik Sense app for troubleshooting Qlik Sense Enterprise on Windows logs

100% helpful (3/3)
cancel
Showing results for 
Search instead for 
Did you mean: 
Mario_Petre
Support
Support

LogAnalysis App: The Qlik Sense app for troubleshooting Qlik Sense Enterprise on Windows logs

Last Update:

Feb 15, 2024 2:53:01 AM

Updated By:

Sonja_Bauernfeind

Created date:

Sep 9, 2022 4:09:13 AM

Attachments

It is finally here: The first public iteration of the Log Analysis app. Built with love by Customer First and Support.

"With great power comes great responsibility." 

Before you get started, a few notes from the author(s):

  • It is a work in progress. Since it is primarily used by Support Engineers and other technical staff, usability is not the first priority. Don't judge. 
  • It is not a Monitoring app. It will scan through every single log file that matches the script criteria and this may be very intensive in a production scenario. The process may also take several hours, depending on how much historical data you load in. Make sure you have enough RAM 🙂
  • Not optimised, still very powerful. Feel free to make it faster for your usecase. 
  • Do not trust chart labels; look at the math/expression if unsure. Most of the chart titles make sense, but some of them won't. This will improve in the future. 
  • MOD IT! If it doesn't do something you need, build it, then tell us about it! We can add it in. 
  • Send us your feedback/scenarios!

Troy_Raney_0-1688711033953.gif

 

Chapters:

  •  01:23 - Log Collector

  • 02:28 - Qlik Sense Services

  • 04:17 - How to load data into the app

  • 05:42 - Troubleshooting poor response times

  • 08:03 - Repository Service Log Level

  • 08:35 - Transactions sheet

  • 12:44 - Troubleshooting Engine crashes

  • 14:00 - Engine Log Level

  • 14:47 - QIX Performance sheets

  • 17:50 - General Log Investigation

  • 20:28 - Where to download the app

  • 20:58 - Q&A: Can you see a log message timeline?

  • 21:38 - Q&A: Is this app supported?

  • 21:51 - Q&A: What apps are there for Cloud?

  • 22:25 - Q&A: Are logs collected from all nodes?

  • 22:45 - Q&A: Where is the latest version?

  • 23:12 - Q&A: Are there NPrinting templates?

  • 23:40 - Q&A: Where to download Qlik Sense Desktop?

  • 24:20 - Q&A: Are log from Archived folder collected?

  • 25:53 - Q&A: User app activity logging?

  • 26:07 - Q&A: How to lower log file size?

  • 26:42 - Q&A: How does the QRS communicate?

  • 28:14 - Q&A: Can this identify a problem chart?

  • 28:52 - Q&A: Will this app be in-product?

  • 29:28 - Q&A: Do you have to use Desktop?

 

Environment

Qlik Sense Enterprise on Windows (all modern versions post-Nov 2019)

 

 

How to use the app:

  1. Go to the QMC and download a LogCollector archive or grab one with the LogCollector tool
  2. Unzip the archive in a location visible to your user profile
  3. Download the attached QVF file
  4. Import/open it in Qlik Sense 
  5. Go to "Data Load Editor" and edit the existing "Logs" folder connection, and point to the extracted Log Collector archive path
    1. If you are using a Qlik Sense server, remember to change the Data Connection name back to default "Logs". Editing via Hub will add your username to the data connection when saved.
  6. Go to the "Initialize" script section and configure:
    1. Your desired date range or days to load
    2. Whether you want the data stored in a QVD
    3. Which Service logs to load (Repository, Engine, Proxy and Scheduler services are built-in right now, adding other Qlik Sense Enterprise services may cause data load errors).
  7. LOAD the data!

 

My workflow:

  1. I'm looking for a specific point in time where a problem was registered
  2. I use the time-based bar charts to find problem areas, get a general sense of workload over time
  3. I use the same time-based charts to narrow in on the problem timestamp
  4. Use the different dimensions to zoom in and out of time periods, down to a per-call granularity
  5. Log Details sheets to inspect activity between services and filter until the failure/error is captured
  6. Create and customise new charts to reveal interesting data points 
  7. Bookmarks for everything!

 

Notable Sheets & requirements:

  • Anything "Thread"-related for analysing Repository Service API call performance, which touches all aspects of the user and governance experience
    • Requirement: Repository Trace Performance logs in DEBUG level. Otherwise, some objects may be empty or broken.
  • Commands: great for visualizing Repository operations and trends between objects, users, and requests
  • Transactions: Repository Service API call performance analysis. 
    • Requirement: Repository Trace Performance logs in DEBUG level. Otherwise, some objects may be empty or broken.
  • Task Transactions: very powerful task scheduling analysis with time-based filters for exclusion. 
  • Log Details sheets: excellent filtering and searching through massive amounts of logs.
  • Repo + Engine data: resource consumption and Thread charts for Repository and Engine services, great for correlating workloads.  

*It is best used in an isolated environment or via Qlik Sense Desktop. It can be very RAM and CPU intensive.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

 

Related Content 

Optimizing Performance for Qlik Sense Enterprise - Qlik Community - 1858594

Comments
QFabian
Specialist III
Specialist III

@Mario_Petre , thanks for this, very clear and usefull

rockabs
Contributor III
Contributor III

It's been almost one year that I am using this application. I love it for our stakeholder that we able to analyze before Qlik help many cases. He released latest version which will give nicely about file lock system traceability.

The way @Mario_Petre always having in detail about his way of working about how that algorithm works that will give so confident like us to explain in smarter way to our entrepreneurs. Appreciate your work!

p_verkooijen
Partner - Specialist
Partner - Specialist

@Mario_Petre  created a new Lib pointing to the logs and script failed 

 
Unexpected token: 'bi_qlik', expected one of: 'biff', 'dif', 'fix', 'html', 'json', 'kml', 'ooxml', ...
 
Concatenate (working) LOAD 'Engine' as Source, 'AuditActivity' as SourceType, FileName() AS [Log Source], * from lib://ServerLogs (>>>>>>bi_qlik<<<<<<)/servername/Engine/Audit/BI_AuditActivity_Engine_2022-04-20T06.10.04Z.log (txt, utf8, embedded labels, delimiter is '\t', no quotes) where

 

Fixed it by enclosing the variable $(vFile) with [] on sheet "File loading"

  • * from $(vFile)
  • * from [$(vFile)]
 
p_verkooijen
Partner - Specialist
Partner - Specialist

@Mario_Petre for somewhat faster processing I changed the following (bold)

We have logfiles dating back to 2018 🙂 now only scanning the relevant year(s) of logfiles

Section: Initialize

//This will load the last x days of logs from the chosen location. Increase value or use specific date range filter above
Set NoOfDays=30;
Let DateFrom = Date(Now() - NoOfDays);
Let DateTo = Date(Now());
Let YearFrom = Year(Now() - NoOfDays);
Let YearTo = Year(Now());

Section: File loading

for vYearLoop = $(YearFrom) to $(YearTo)
 
      for each vFile in FileList('$(dir)/*$(vYearLoop)*.$(Ext)')
 
next vFile;
 
next vYearLoop
set vYearLoop;
makunii
Partner - Contributor III
Partner - Contributor III

Hello @Mario_Petre 

I downloaded the Log Analyzer app but didn't see the QIX Performance and QIX Performance 2 sheet.

Thank you

Regards,

Marco

sujeet_shirude
Creator II
Creator II

Hello @Mario_Petre;

@Troy_Raney ;

@Sonja_Bauernfeind 

I ran into the same issue as @makunii I downloaded the Log Analyzer app but didn't see the QIX Performance and QIX Performance 2 sheet.

Any chance we can get the latest version of the app that has QIX Performance and QIX Performance 2 sheet?

Thanks in advance!

Kalkumar
Partner - Contributor III
Partner - Contributor III

@Mario_Petre  cannot see QIX Performance and QIX Performance 2 sheet.

Sonja_Bauernfeind
Digital Support
Digital Support

Hello @Kalkumar 

I've attached the version you are looking for. I've also replaced the previous one with an even more recent one now. Both include the QIX sheets.

All the best,
Sonja 

Version history
Last update:
2 weeks ago
Updated by: