Qlik Community

Knowledge

Search or browse our knowledge base to find answers to your questions ranging from account questions to troubleshooting error messages. The content is curated and updated by our global Support team

Announcements
Join us for live office hours! Q&A with Qlik on Dec 14 at 10am EST: Refining Reports and Visualizations

How to decrypt Qlik Replicate verbose task log files

john_wang
Support
Support

How to decrypt Qlik Replicate verbose task log files

Product Version

  • Qlik Replicate November 2021 (Version 2021.11).

 

What's Replicate log files encryption

Starting from Qlik Replicate version "November 2021", when the logging level is set to Verbose, sections in a Replicate log file that might contain customer data will be encrypted while the rest of the log will remain in clear text, this is implemented by the enable_data_logging configuration parameter was set to false by default in order to prevent customer data snippets from appearing in verbose logs.

The sample line of the task log file looks like:

2021-11-05T15:45:11:390848 [AT_GLOBAL       ]I:  Logging of database data is disabled  (at_logger.c:2702)

 

However in some support cases, R&D may require the ability to decrypt the log in order to analyze the problem. In such cases, Qlik Support will ask you to provide the task or environment-level encryption key and corresponding task log files:

  • Replicate task diagnostics package file, and
  • log.key file for decrypting a task's log file, location is <REPLICATE_INSTALL_DIR>\data\tasks\<task-name>
  • log.key file for decrypting non-task log files (e.g. server logs), location is <REPLICATE_INSTALL_DIR>\data


Below is the detailed steps how to enable the database data logging and decrypt a task log file with corresponding log.key files.

 

Steps

    1. Stop the Replicate Server service.

    2. Open <REPLICATE_INSTALL_DIR>\bin\repctl.cfg and set the enable_data_logging parameter to true. (adds the line if it does not exist)

{
	"port": 3552,
	"plugins_load_list": "repui",
... ...
	"enable_data_logging": true
}

    3. Save the repctl.cfg file and start the Replicate Server service.

    4. Now the task log files contains encrypted database data, sample lines like:

2021-11-05T15:59:07:969146 [AT_GLOBAL       ]I:  Log file encryption base "755opF/9oGw7YOx+2ma0dA=="  (at_logger.c:2695)
2021-11-05T15:59:07:969146 [INFRASTRUCTURE  ]I:  The log level for 'TARGET_APPLY' has been changed from 'INFO' to 'VERBOSE'.  (at_logger.c:2927)
2021-11-05T15:59:07:969146 [INFRASTRUCTURE  ]I:  The log level for 'SOURCE_CAPTURE' has been changed from 'INFO' to 'VERBOSE'.  (at_logger.c:2927)

... ...

2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  Event of captured table 'SCOTT.KIT' (data object id '93719', object id '93719')  (oradcdc_parse.c:303)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  xid [000008fb00100007]  (oradcdc_parse.c:166)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  BI col  2 [5]
~uAAAABvF/duclauaOYlVdTbsCpbAiSZGsq3DIp8XyImAZjY984KVgc62Lir20Mnw1VU1V/54j/vXa4BXpyQ7rjcEPR1MtjoYnpPSeu4QSwfiUx3l}~  (oradcdc_parse.c:340)

         The latest line is the encrypted database data (BeforeImage of an update).

    5. Run "Qlik Replicate Command Line" prompt "as administrator", and change to directory <REPLICATE_INSTALL_DIR>\bin

    6. Run the following command to decrypt task logs:

repctl dumplog <log-file-path> <log.key-path> [> <path-to-output-file>]

         A sample command:

repctl dumplog "C:\Program Files\Attunity\Replicate\data\logs\reptask_Ora-to-SQL-2.log" "C:\Program Files\Attunity\Replicate\data\tasks\Ora-to-SQL-2\log.key" > "C:\Program Files\Attunity\Replicate\data\logs\reptask_Ora-to-SQL-2-dump.log"

         Now we see the decrypted data in the task log files. The latest line is the BeforeImage of an update in clear text (in this sample it's text string "xxxxx") :

2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  Event of captured table 'SCOTT.KIT' (data object id '93719', object id '93719')  (oradcdc_parse.c:303)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  xid [000008fb00100007]  (oradcdc_parse.c:166)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE  ]V:  BI col  2 [5]
25a058b66d8: 7878787878                       | xxxxx           
  (oradcdc_parse.c:340)

 

Related Instructions

The encryption key file (log.key file) can be used to decrypt any log that was encrypted with it, not just the log for which Qlik Support requested the encryption key file. If this is a concern, you can generate a log.key file that will only be valid for the specific log requested by Qlik Support.
To do this:
    1. Delete the log.key file.
    2. Restart the task or Replicate Server service (depending on the log type required) to generate a new log.key file.
    3. Send the requested log file and the log.key file to Qlik Support.
    4. Delete the log.key file.
    5. Repeat steps 2-4 if you need to provide additional encrypted logs.

 

Products: Qlik Replicate 

Version history
Last update:
2 weeks ago
Updated by:
Contributors