Qlik Community

Qlik Design Blog

All about product and Qlik solutions: scripting, data modeling, visual design, extensions, best practices, etc.

Announcements
Read about the latest Qlik Community enhancements on the Community News blog!
Michael_Tarallo
Employee
Employee

In an enterprise environment IT does not give access to the folders where the logs are stored, so developers have to ask to get the logs from IT which can be terribly slow and frustrating.

andrea.pngHey guys - today I would like to introduce you to our latest guest blogger, Andrea Danielato.  Andrea has been with Qlik for 10 years and is a Senior Consultant. He has worked on customer projects acting as a developer, a project manager and an advisor for Qlik solutions. He has helped customers optimize app performance, define and implement best practices and infrastructure governance. He is heavily involved with teaching our valued partners how to deliver the best Qlik applications. On a personal note, Andrea likes to watch movies and loves to practice DIY. Please welcome me in joining Andrea to the guest blogger program - where today he will provide you with a solution to read RELOAD LOGS (attached).

Take it away Andrea!

reload.gif

Andrea:

Hello Qlik Community! You will be pleased to know that I have developed a solution for developers, that helps with access to Qlik Sense app reload logs for debugging purposes.

In an enterprise environment, it may be common where IT will not provide access to the file structure / folders where the logs are stored, so developers have to ask to get the logs from IT which can be terribly slow and frustrating.

To solve this I have developed a simple ODAG solution that can help on this topic.

I attach a document with a short manual of the solution including installation and a short vide that demonstrates the functionality of the solution.

Let me know what you think!

 

8 Comments
Or
Master
Master

Interesting! I've got a similar app built which reads the logs (including the log text) from the reload logs folder, as well as creating stripped-down versions without the timestamps so the SQL / load scripts can be copied by those don't have access to the script itself. It also means we can search for specific fields/code snippets from within QS itself.

 

Looks like there's stuff you're doing I could stand to learn from to improve on my version (or replace it outright). Thanks for sharing!

 

One added note - with the May 2021 QS release, it's possible for people to download the last log for each task directly from hub, assuming they're assigned the correct security rules and have access to the app itself, which also helps with giving people access to logs without having to give them direct access to the folders. Previously this could only be done by giving them QMC access to the Tasks screen, which was also an option but more cumbersome.

1,365 Views
adt
Employee
Employee

Thanks Or for your post.

The possibility to download the log from hub can be, from my experience, breaking some internal rules at customer side especially in Enterprise environment.

Anyway using this approach you can get only the last log and not the oldest ones where you can have additional details like a stopped execution and you can compare the timing to complete the reload.

 

 

 

1,145 Views
Or
Master
Master

@adt  Agreed - different use cases require different types of information, of course, and like I said your app has a lot I can learn from - I never thought to use ODAG for log reading.

Now if only we could get Qlik to have a setting that saves the last X logs, but also the last X logs which ended in an error, rather than only saving the last X (default is 4, I think) and then we lose track of the errors with apps scheduled to run frequently (or we save way too many logs). But that's for Ideation, I suppose.

0 Likes
979 Views
adt
Employee
Employee

@Or good point. We can post an idea on the ideation site about the last X logs but in the meantime I will make some thinking about checking the logs with error and save them in QVD for future investigation, with a time flag to delete the most old ones.

Thanks, I will add this to the next version!

Andrea

954 Views
mountaindude
Luminary Alumni
Luminary Alumni

ODAG can indeed be used for all kinds of things - digging into logs probably wasn't one of the expected use cases - very useful nonetheless!!

If you want to keep track of your reloads and what caused them to fail, the Butler's alert emails can also be useful. 
Indeed, I am the creator of that tool, but as it's related but not competing I figured it's ok to mention it here.

The idea is that reload failures are tracked by means of a hook into Qlik Sense's logging framework, then forwarded to Butler. There the event is enriched with metadata such as reload logs, then sent as an email, Slack or Teams message, or as a webhook. 

The really nice part is that the alert email/Slack/Teams message can include (if so desired) the last few (again, configurable) rows in the reload log.
From there it's usually pretty easy to determine what caused the reload failure.

It can look like this:

mountaindude_0-1630440494832.png

Main Butler site is available at https://butler.ptarmiganlabs.com. Open source, of course.

Cheers,
Göran

933 Views
adt
Employee
Employee

@mountaindude very interesting approach to a common problem for developers and IT application management groups.

The only point that I can see in large enterprise is breaking security rules defined by the IT department. Generally asking to install something on a governed server is always a big fighting and a lot of discussion.

Anyway a very good idea of using different alerts systems to share the interesting log part.

Andrea

0 Likes
917 Views
Levente_Szittya
Partner
Partner

Hi,

Could you please advise what shall be done with this error?

The dataconnection in the imported app does not work. I have tried to create a "PostgreSQL" data connection, but I do not know exactly what and where should I look for?

Thanks,

Levente

Screenshot 2021-10-05 112259.jpg

 

0 Likes
211 Views
adt
Employee
Employee

Hi Levente,

the Postgress_Idra library is a data connection created using the PostgreSQL data driver that point to the Qlik Sense database definited during the installation of Qlik Sense. You can ask to the IT that has managed the installation to get the parameters for the Postgere database, user and password.

Andrea

adt_0-1633863748212.png

 

139 Views