Appender for log files in qlik sense

[#1]

Hi,

I am trying to configure the custom log appender within qlik sense following the directions specified here:

Local log configuration file ‒ Qlik Sense

In this help (unhelpful) file it says to add a new file called this:

LocalLogConfig.xml

into a directory (see below)

To set up customized logging, create a local log configuration file named LocalLogConfig.xml in the %ProgramData%\Qlik\Sense\<Service>\ folder.

Does anyone know what the <Service> part is? Is this a new folder? I've gone into this directory but only see the following folders

Apps

Engine

Installation

Log

Proxy

Repository

Scheduler

Also where it says service does that mean you need a separate LocalLogConfig.xml per service?

Thanks,

Lee

[Bjorn_Wedbratt]

Hi Lee,

That's correct, you need to create the LocalLogConfig.xml file in the folder(s) for each of the service(s) you want to configure to use an appender.

So for example, if you want to use an SMTPAppender to send an email whenever a task is failing, you need to create the LocalLogConfig.xml file in %ProgramData%\Qlik\Sense\Scheduler as it is the Scheduler service you want to monitor for any errors on failing tasks.

Hope the above clarifies it a bit

Best,

Bjorn

[#3]

Hi Bjorn,

Thanks for your reply. I am still a bit confused as to where to add the LocalLogConfig.xml should this be

%ProgramData%\Qlik\Sense\Sense\Log\Scheduler\

or

%ProgramData%\Qlik\Sense\Sense\Scheduler\

I've tried both but they don't seem to work.

I am trying to configure a AdoNetAppender appender to write anything from the scheduler directly to a db table so have been using the below definition for the LocalLogConfig.xml. Not sure if this is correct or if I should be using this element at all? <logger name="Audit.Proxy"><appender-ref ref="AdoNetAppender" /></logger>. If you have any ideas that would be great.

<?xml version="1.0"?>

<configuration>

<appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">

    <bufferSize value="100" />

    <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />

    <connectionString value="data source=[the database server];initial catalog=[the database];integrated security=true" />

    <commandText value="INSERT INTO qliksense.Log ([Date],[Thread],[Level],[Logger],[Message],[Exception]) VALUES (@log_date, @thread, @log_level, @logger, @message, @exception)" />

    <parameter>

        <parameterName value="@log_date" />

        <dbType value="DateTime" />

        <layout type="log4net.Layout.RawTimeStampLayout" />

    </parameter>

    <parameter>

        <parameterName value="@thread" />

        <dbType value="String" />

        <size value="255" />

        <layout type="log4net.Layout.PatternLayout">

            <conversionPattern value="%thread" />

        </layout>

    </parameter>

    <parameter>

        <parameterName value="@log_level" />

        <dbType value="String" />

        <size value="50" />

        <layout type="log4net.Layout.PatternLayout">

            <conversionPattern value="%level" />

        </layout>

    </parameter>

    <parameter>

        <parameterName value="@logger" />

        <dbType value="String" />

        <size value="255" />

        <layout type="log4net.Layout.PatternLayout">

            <conversionPattern value="%logger" />

        </layout>

    </parameter>

    <parameter>

        <parameterName value="@message" />

        <dbType value="String" />

        <size value="4000" />

        <layout type="log4net.Layout.PatternLayout">

            <conversionPattern value="%message" />

        </layout>

    </parameter>

    <parameter>

        <parameterName value="@exception" />

        <dbType value="String" />

        <size value="2000" />

        <layout type="log4net.Layout.ExceptionLayout" />

    </parameter>

</appender>

  <logger name="Audit.Proxy">

      <appender-ref ref="AdoNetAppender" />

  </logger>

</configuration>

Thanks,

Lee

[Bjorn_Wedbratt · Accepted solution]

Hi Lee,

The LocalLogConfig.xml should go into %ProgramData%\Qlik\Sense\Sense\Scheduler\ and not into the \log folder, to answer your first question.

I haven't played around with the ADO.NET Appender so I cannot say how to configure the specifics for that appender (seems like what you have is accurate according to log4net documentation though).

What you could do is to try finding out the root cause for it failing, by debugging log4net (basically activate logging for the logger).

This can be done by adding the following into the .exe.config file for the service, in your case the file will be:

%ProgramFiles%\Qlik\Sense\Scheduler\Scheduler.exe.config

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

    <appSettings>

        <add key="log4net.Internal.Debug" value="true"/>

    </appSettings>

    <system.diagnostics>

        <trace autoflush="true">

            <listeners>

                <add

                    name="textWriterTraceListener"

                    type="System.Diagnostics.TextWriterTraceListener"

                    initializeData="C:\files\log4net.txt" />

            </listeners>

        </trace>

    </system.diagnostics>

</configuration>

So in <appSettings> tag you add a key to enable log4net debugging and in <system.diagnostics> (you need to add this section) you add a textwriter to flush out the debugging information into a file (above file will be generated in C:\files which must exist)

Make sure not to delete or alter any other sections within the config file or the service may fail to start

Investigating the log4net.txt log may give you hints on where it's failing.

Just a hint. If I got time I will see if I can try it out using the ADO.NET Appender as well, but cannot promise anything in the near future.

Best,

Bjorn

[#5]

Hi Bjorn,

Many thanks for your reply, adding the logging for the log4net logging I was able to track down that the logging config setup in LocalLogConf.xml was not being read as there were no entries in the debug log.

To fix this I added the following to the service config file and created the new file LocalLog.config to store the config for the log4net.

<configSections>

      <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>

</configSections>

<appSettings>

    <add key="log4net.Config" value="LocalLog.config" />

</appSettings>

Logging still didn't work after doing this but it did register the appender in the log4net debug log.

In the end the final issue turned out to be this element in the LocalLog.config

<logger name="AuditActivity.Scheduler">

     <appender-ref ref="AdoNetAppender" />

</logger>

which was changed as follows:

<?xml version="1.0"?>

<log4net>

<appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">

.....

</appender>

<root>

  <appender-ref ref="AdoNetAppender" />

</root>

</log4net>

The logging now works as expected and I'm storing this in the database.

Note another hint I've picked up from reading various pages is to change the buffer size to 1 from 100 so that entries are flushed as they occur.

Thanks,

Lee

[adambrian]

Hi,

I new in Qlik Sense, can someone tell me the different between LocalLogConfig.xml and Scheduler.exe.config ? Are both of them are same?

Thanks,

Adam

[#7]

Hi Adam,

As I understand it the Scheduler.exe.config is the configuration for the scheduler service. The LocalLogConfig.xml is the configuration for the log4net appender.

Thanks,

Lee

[adambrian]

Hi Lee,

Thank you for your info.

Thanks,

Adam

[wallace0834]

I have followed all of the steps provided with the LocalXMLconf file with my stmp server, but I still don't get an e-mail notification when task failed.  I have placed this file in the C:\ProgramData\Qlik\Sense\Scheduler.  Please let me know if I need to change or alter the schedule.exe.config file.