7 Replies Latest reply: Aug 3, 2016 2:23 AM by Adam Brian RSS

    Appender for log files in qlik sense

    Lee Young

      Hi,

       

      I am trying to configure the custom log appender within qlik sense following the directions specified here:

       

      Local log configuration file ‒ Qlik Sense

       

      In this help (unhelpful) file it says to add a new file called this:

       

      LocalLogConfig.xml

       

      into a directory (see below)

       

      To set up customized logging, create a local log configuration file named LocalLogConfig.xml in the %ProgramData%\Qlik\Sense\<Service>\ folder.

       

      Does anyone know what the <Service> part is? Is this a new folder? I've gone into this directory but only see the following folders

       

      Apps

      Engine

      Installation

      Log

      Proxy

      Repository

      Scheduler

       

      Also where it says service does that mean you need a separate LocalLogConfig.xml per service?

       

      Thanks,

       

      Lee

       

       

        • Re: Appender for log files in qlik sense
          Björn Wedbratt

          Hi Lee,

          That's correct, you need to create the LocalLogConfig.xml file in the folder(s) for each of the service(s) you want to configure to use an appender.

          So for example, if you want to use an SMTPAppender to send an email whenever a task is failing, you need to create the LocalLogConfig.xml file in %ProgramData%\Qlik\Sense\Scheduler as it is the Scheduler service you want to monitor for any errors on failing tasks.

           

          Hope the above clarifies it a bit

           

          Best,

          Bjorn

            • Re: Appender for log files in qlik sense
              Lee Young

              Hi Bjorn,

               

              Thanks for your reply. I am still a bit confused as to where to add the LocalLogConfig.xml should this be


              %ProgramData%\Qlik\Sense\Sense\Log\Scheduler\

              or

              %ProgramData%\Qlik\Sense\Sense\Scheduler\

               

              I've tried both but they don't seem to work.


              I am trying to configure a AdoNetAppender appender to write anything from the scheduler directly to a db table so have been using the below definition for the LocalLogConfig.xml. Not sure if this is correct or if I should be using this element at all? <logger name="Audit.Proxy"><appender-ref ref="AdoNetAppender" /></logger>. If you have any ideas that would be great.

               

              <?xml version="1.0"?>

              <configuration>

              <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">

                  <bufferSize value="100" />

                  <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />

                  <connectionString value="data source=[the database server];initial catalog=[the database];integrated security=true" />

                  <commandText value="INSERT INTO qliksense.Log ([Date],[Thread],[Level],[Logger],[Message],[Exception]) VALUES (@log_date, @thread, @log_level, @logger, @message, @exception)" />

                  <parameter>

                      <parameterName value="@log_date" />

                      <dbType value="DateTime" />

                      <layout type="log4net.Layout.RawTimeStampLayout" />

                  </parameter>

                  <parameter>

                      <parameterName value="@thread" />

                      <dbType value="String" />

                      <size value="255" />

                      <layout type="log4net.Layout.PatternLayout">

                          <conversionPattern value="%thread" />

                      </layout>

                  </parameter>

                  <parameter>

                      <parameterName value="@log_level" />

                      <dbType value="String" />

                      <size value="50" />

                      <layout type="log4net.Layout.PatternLayout">

                          <conversionPattern value="%level" />

                      </layout>

                  </parameter>

                  <parameter>

                      <parameterName value="@logger" />

                      <dbType value="String" />

                      <size value="255" />

                      <layout type="log4net.Layout.PatternLayout">

                          <conversionPattern value="%logger" />

                      </layout>

                  </parameter>

                  <parameter>

                      <parameterName value="@message" />

                      <dbType value="String" />

                      <size value="4000" />

                      <layout type="log4net.Layout.PatternLayout">

                          <conversionPattern value="%message" />

                      </layout>

                  </parameter>

                  <parameter>

                      <parameterName value="@exception" />

                      <dbType value="String" />

                      <size value="2000" />

                      <layout type="log4net.Layout.ExceptionLayout" />

                  </parameter>

              </appender>

                <logger name="Audit.Proxy">

                    <appender-ref ref="AdoNetAppender" />

                </logger>

              </configuration>

               

              Thanks,

               

              Lee

                • Re: Appender for log files in qlik sense
                  Björn Wedbratt

                  Hi Lee,

                  The LocalLogConfig.xml should go into %ProgramData%\Qlik\Sense\Sense\Scheduler\ and not into the \log folder, to answer your first question.


                  I haven't played around with the ADO.NET Appender so I cannot say how to configure the specifics for that appender (seems like what you have is accurate according to log4net documentation though).


                  What you could do is to try finding out the root cause for it failing, by debugging log4net (basically activate logging for the logger).


                  This can be done by adding the following into the .exe.config file for the service, in your case the file will be:

                  %ProgramFiles%\Qlik\Sense\Scheduler\Scheduler.exe.config


                  <?xml version="1.0" encoding="utf-8" ?>

                  <configuration>

                      <appSettings>

                          <add key="log4net.Internal.Debug" value="true"/>

                      </appSettings>

                      <system.diagnostics>

                          <trace autoflush="true">

                              <listeners>

                                  <add

                                      name="textWriterTraceListener"

                                      type="System.Diagnostics.TextWriterTraceListener"

                                      initializeData="C:\files\log4net.txt" />

                              </listeners>

                          </trace>

                      </system.diagnostics>

                  </configuration>


                  So in <appSettings> tag you add a key to enable log4net debugging and in <system.diagnostics> (you need to add this section) you add a textwriter to flush out the debugging information into a file (above file will be generated in C:\files which must exist)

                  Make sure not to delete or alter any other sections within the config file or the service may fail to start


                  Investigating the log4net.txt log may give you hints on where it's failing.

                  Just a hint. If I got time I will see if I can try it out using the ADO.NET Appender as well, but cannot promise anything in the near future.

                   

                  Best,

                  Bjorn

                   

                    • Re: Appender for log files in qlik sense
                      Lee Young

                      Hi Bjorn,

                       

                      Many thanks for your reply, adding the logging for the log4net logging I was able to track down that the logging config setup in LocalLogConf.xml was not being read as there were no entries in the debug log.

                       

                      To fix this I added the following to the service config file and created the new file LocalLog.config to store the config for the log4net.

                       

                      <configSections>

                            <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>

                      </configSections>

                       

                      <appSettings>

                          <add key="log4net.Config" value="LocalLog.config" />

                      </appSettings>

                       

                      Logging still didn't work after doing this but it did register the appender in the log4net debug log.

                       

                      In the end the final issue turned out to be this element in the LocalLog.config

                       

                      <logger name="AuditActivity.Scheduler">

                           <appender-ref ref="AdoNetAppender" />

                      </logger>

                       

                      which was changed as follows:

                       

                      <?xml version="1.0"?>

                      <log4net>

                      <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">

                       

                      .....

                       

                      </appender>

                      <root>

                        <appender-ref ref="AdoNetAppender" />

                      </root>

                      </log4net>

                       

                      The logging now works as expected and I'm storing this in the database.

                       

                      Note another hint I've picked up from reading various pages is to change the buffer size to 1 from 100 so that entries are flushed as they occur.

                       

                      Thanks,

                       

                      Lee