Environment
- Qlik Replicate November 2021 (Version 2021.11 and later)
What is Replicate log file encryption?
Starting from Qlik Replicate version "November 2021", when the logging level is set to Verbose, sections in a Replicate log file that might contain customer data will be encrypted while the rest of the log will remain in clear text, this is implemented by the enable_data_logging configuration parameter set to true by default in order to prevent customer data snippets from appearing in verbose logs.
The sample line of the task log file looks like this:
2021-11-05T15:45:11:390848 [AT_GLOBAL ]I: Logging of database data is disabled (at_logger.c:2702)
However, in some support cases, R&D may require the ability to decrypt the log in order to analyze the problem. In such cases, Qlik Support will ask you to provide the task or environment-level encryption key and corresponding task log files:
- Replicate task diagnostics package file, and
- log.key file for decrypting a task's log file, location is <REPLICATE_INSTALL_DIR>\data\tasks\<task-name>
- log.key file for decrypting non-task log files (e.g. server logs), location is <REPLICATE_INSTALL_DIR>\data
Below is the detailed steps how to enable the database data logging and decrypt a task log file with corresponding log.key files.
Steps
- Stop the Replicate Server service.
- Open <REPLICATE_INSTALL_DIR>\bin\repctl.cfg and set the enable_data_logging parameter to true. (add the line if it does not exist). This is set to true by default in version 2021.11 and later, even if you do not see this line in your configuration file.
{
"port": 3552,
"plugins_load_list": "repui",
... ...
"enable_data_logging": true
}
- Save the repctl.cfg file and start the Replicate Server service.
- Now the task log files contain encrypted database data, sample lines like:
2021-11-05T15:59:07:969146 [AT_GLOBAL ]I: Log file encryption base "755opF/9oGw7YOx+2ma0dA==" (at_logger.c:2695)
2021-11-05T15:59:07:969146 [INFRASTRUCTURE ]I: The log level for 'TARGET_APPLY' has been changed from 'INFO' to 'VERBOSE'. (at_logger.c:2927)
2021-11-05T15:59:07:969146 [INFRASTRUCTURE ]I: The log level for 'SOURCE_CAPTURE' has been changed from 'INFO' to 'VERBOSE'. (at_logger.c:2927)
... ...
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: Event of captured table 'SCOTT.KIT' (data object id '93719', object id '93719') (oradcdc_parse.c:303)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: xid [000008fb00100007] (oradcdc_parse.c:166)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: BI col 2 [5]
~uAAAABvF/duclauaOYlVdTbsCpbAiSZGsq3DIp8XyImAZjY984KVgc62Lir20Mnw1VU1V/54j/vXa4BXpyQ7rjcEPR1MtjoYnpPSeu4QSwfiUx3l}~ (oradcdc_parse.c:340)
The latest line is the encrypted database data (BeforeImage of an update).
- Run "Qlik Replicate Command Line" prompt "as administrator", and change to directory <REPLICATE_INSTALL_DIR>\bin
- Run the following command to decrypt task logs:
repctl dumplog <log-file-path> <log.key-path> [> <path-to-output-file>]
A sample command:
repctl dumplog "C:\Program Files\Attunity\Replicate\data\logs\reptask_Ora-to-SQL-2.log" "C:\Program Files\Attunity\Replicate\data\tasks\Ora-to-SQL-2\log.key" > "C:\Program Files\Attunity\Replicate\data\logs\reptask_Ora-to-SQL-2-dump.log"
Now we see the decrypted data in the task log files. The latest line is the BeforeImage of an update in clear text (in this sample it is text string "xxxxx"):
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: Event of captured table 'SCOTT.KIT' (data object id '93719', object id '93719') (oradcdc_parse.c:303)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: xid [000008fb00100007] (oradcdc_parse.c:166)
2021-11-05T16:02:21:346746 [SOURCE_CAPTURE ]V: BI col 2 [5]
25a058b66d8: 7878787878 | xxxxx
(oradcdc_parse.c:340)
Related Instructions
The encryption key file (log.key file) can be used to decrypt any log that was encrypted with it, not just the log for which Qlik Support requested the encryption key file. If this is a concern, you can generate a log.key file that will only be valid for the specific log requested by Qlik Support.
To do this:
- Delete the log.key file.
- Restart the task or Replicate Server service (depending on the log type required) to generate a new log.key file.
- Send the requested log file and the log.key file to Qlik Support.
- Delete the log.key file.
- Repeat steps 2-4 if you need to provide additional encrypted logs.
If you do not want verbose logs encrypted, open <REPLICATE_INSTALL_DIR>\bin\repctl.cfg and set the enable_data_logging parameter to false then restart the Replicate server service. Add the line if you do not see it in the file.
Products: Qlik Replicate
