How to analyze a Qlik Replicate log

[OritA]

This article recommends a path on how to analyze a Qlik Replicate log. 

Verify your Qlik Replicate version

The Qlik Replicate version appears at the first line of the log. In the example image below, the Qlik Replicate version is 2021.11 build 469 (i.e. PR06). 

The name of the Qlik Replicate server and its platform type are displayed immediately after the version. In our example, the Qlik Replicate server's name is STAM.qliktech.com and it is running on a Windows platform.

If you are running an old or unsupported Qlik Replicate version or a GA release (or even an early service pack), it is likely that any defects you are experiencing are resolved in the latest version. We recommend crosschecking issues with defects fixed in our release notes. 

Check whether the task is Full load or CDC or both

For many problems it is important to know what type of task is being run. For example, are we lookin gat a Full load task or a CDC task? 

This information will usually appear near the first lines of the log.

To locate them quckly, search for the word running.  

You can also get this information from the task JDSON which is included in the task diagnostic package.

In the example below the task named ‘s-2-s’ is Full load and CDC task.

How was the task started?

Was it started with fresh start (fig started)?

started

Was processing resumed (fig resumed)?

resumed

Was processing resumed from timestamp (fig resume timestamp)?

resume timestamp

Was the task resumed after recoverying from an error (fig resume error)?

resume error

This information is relevant if a task was resumed after a recoverable error. This means you will need to look for the original error that caused the resume for recovery in the previous log file. 

Search for errors and warnings

Every log line has a generic structure:

• It starts with a number representing the number of the Replicate thread that generated the log message
• Followed by date and time (when the log line was generated)
• Includes the name of the Qlik Replicate logger component that generated the log event. For more information on the Qlik Replicate logger components, see Replicate Loggers 
• Indicates the message level (for more information, see: How to read a Qlik Replicate log)
  
  
  • Error - marked in the log with ]E:
  • Warning - marked in the log with ]W:
  • Info - marked in the log with ]I:
  • Trace - marked in the log with ]T:
  • Verbose - marked in the log with ]V:
    
    

When starting to look at the log in order to troubleshoot a problem, it is recommended to search first for errors (you can simply search the string ]E:)  and also search for warnings (you can simply search the string ]W:).

Example Error:

In the example below you can see that the task failed with a DB2 error -803 (SQL0803N) when trying to apply the insert command listed in the first line on the target endpoint (the Qlik Replicate component that generated the message is TRAGET_APPLY). Since all the lines listed in the example below belong to same Replicate thread (thread # 00053118) we can conclude that they are all related.

Example Warning:

Based on the error\warnings found in the log, you will be able to decide how to continue troubleshooting.

How to turn on detailed logging level?

The logging level of a Qlik Replicate component can be changed in the task settings. 

1. Open the task.
2. Navigate to Logging
3. Click Logging Level 
4. Choose your desired level.
   
   

When to turn on detailed logging levels.

Sometimes you will need additinoal information on what action was performed by Qlik Replicate before an error occurres. For that purpose, you can turn up the Qlik Replicate logging level. For example, we may increase it to TRACE or VERBOSE for the specific Qlik Replicate component of interest. This would be the component that generated the error/warning.

For example, when turning on TRACE logging level, on SOURCE_CAPTUER and/or TARGET_APPLY,  Qlik Replicate will display in the log the SQL commands that it performed to the specific endpoint. The example below shows few of the TRACE log records for SOURCE_CAPTURE when working with Oracle source end point.

Keep in mind that a detailed logging level will produce larger log files.

Environment:

Qlik Replicate 

Related Content:

Qlik Replicate: Example error codes sent by data sources in  

[Dana_Baldwin]

Great article! I have also found it helpful to search for "error" or "fail" in case an endpoint error code is not interpreted by Replicate as needing to be captured with the "]E" error notation. Also, searching for the format of an endpoint error code can be helpful, such as "ORA-" if you are using Oracle.

[eksmirnova]

Thank you, this is a really helpful article. 

Is there an article which describes Component Logging Categories such as FILE_FACTORY, ASSERTION, INFRASTRUCTURE and etc? It is not always obvious which category to set to TRACE/VERBOSE to analyze the issue.

For example, if I am chasing down the issue converting csv file to parquet (]E: Failed to convert file from csv to parquet)  should I set FILE_FACTORY to VERBOSE or TARGET_APPLY?

[Dana_Baldwin]

Hi @eksmirnova 

Yes, this is documented in our user guide: Replicate Loggers | Qlik Replicate Help

For your example, I would try File_Factory, IO and Target_Apply. If the issue happens during the full load phase of a task, rather than change processing, substitute Target_Load for Target_Apply.

Thanks,

Dana

[eksmirnova]

Thank you Dana!