Qlik Replicate May 2023 Release Notes - Initial Release until Service Release 1

Charlton_Book · Apr 20, 2023 4:54:25 PM

What's new
Migration and upgrade
Upgrade paths
Microsoft .NET Framework 4.8 upgrade/installation prerequisite
Deleting the Replicate self-signed certificate after upgrade
Upgrading an installation with multiple data folders
Compatibility with related products
End of life/support features
End of support
Newly supported versions and third-party software
Source endpoint versions
Target endpoint versions
Resolved issues
Resolved issues in May 2023 service release 1
Resolved issues in Replicate May 2023 initial release
Known issues
Downloads

What's new

Learn about new features and improvements in Qlik Replicate.

Migration and upgrade

This section describes the issues that you may encounter when upgrading/migrating to the new version.

Upgrade paths

Qlik supports four Replicate versions (including this one).

Direct upgrade is supported from the last two versions only (including any service releases for those versions). The last two versions are are Replicate May 2022 and Replicate November 2022.

Upgrading from Replicate April 2020 (6.6)

If you are upgrading from Replicate April 2020 (6.6), you need to perform the following upgrades:

Upgrade to Replicate November 2020 (7.0).
Upgrade to Replicate November 2021.
Upgrade to Replicate November 2022.
Upgrade to Replicate May 2023 SR1.

Upgrading from unsupported versions

If you are upgrading from Replicate 6.x (starting from 6.3), you need to perform the following upgrades:

Upgrade to Replicate 6.5.
Upgrade to Replicate November 2020 (7.0).
Upgrade to Replicate November 2021.
Upgrade to Replicate November 2022.
Upgrade to Replicate May 2023 SR1.

Note: If you are upgrading from Replicate 5.5, please contact Qlik Support.

Microsoft .NET Framework 4.8 upgrade/installation prerequisite

Using the Setup Wizard - It is preferable for .NET Framework 4.8 to be installed on the Replicate Server machine before running Setup. If .NET Framework 4.8 is not present on the machine, Setup will prompt you to install it. This may require the machine to be rebooted once the installation completes.
Silent Installation – When recording an Install Shield Silent response file (ISS), the recording must be done on a machine with .NET 4.8 already installed on it as the silent installation process does not include a separate installation of .NET framework 4.8.

Deleting the Replicate self-signed certificate after upgrade

After upgrading, customers that are using Replicate's self-signed certificate (i.e. instead of their own certificate) should perform the following procedure:

Note: This procedure does not need to be performed when upgrading from Replicate November 2022.

Delete all *.pem files from <replicate_data_folder>/ssl/data.
Restart the Qlik Replicate Server service.

This will cause Replicate to generate a new self-signed certificate, thereby ensuring that the certificate will be accepted by newer clients (browsers) that select the more secure TLS 1.3 protocol.

Note that if you do not perform the above procedure, the following error will be encountered when connecting to Replicate Console:

SYS,GENERAL_EXCEPTION,The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

Upgrading an installation with multiple data folders

When upgrading a Replicate installation with multiple Data folders, only the default Data folder (<Product_Dir>\Data) will be automatically upgraded. The other Data folders need to be updated manually by running the following command:

repuictl.exe -d <data_folder_path> setup install

Compatibility with related products

This version of Replicate is compatible with the following related products only:

Enterprise Manager May 2023 or later
Compose May 2022 or later
Compose for Data Lakes April 2020 (6.6)

End of life/support features

This section provides information about End of Life versions, End of Support features, and deprecated features.

End of support

Endpoint versions

Support for the following source endpoint versions has been discontinued:

PostgreSQL 10
MySQL 5.6
Netezza 6.x
Teradata 15.x and 16.x
MariaDB 10.3

Support for the following target endpoint versions has been discontinued:

PostgreSQL 10
MySQL 5.6
Netezza 6.x
Teradata 15.x and 16.x
MariaDB 10.3
Databricks 9.1 LTS

Platforms

Red Hat 7.x: Support for installing on Red Hat 7.x has been discontinued. The minimum supported version is Red Hat 8.x.
Amazon Linux 2: Support for Amazon Linux 2 has been discontinued.
Windows Server 2012: Support for installing Replicate on Windows Server 2012 (64-bit) and Windows Server 2012 R2 (64-bit) will be discontinued from Replicate November 2023.

XML configuration files

The ServiceConfiguration.xml and ServerConfiguration.xml files have been deprecated. Operations that were previously performed by editing these files are now performed using the Replicate CLI. For details about the new commands, refer to the following Replicate topics:

Newly supported versions and third-party software

This section lists the newly supported database versions and third-party software versions.

Source endpoint versions

The following source endpoint versions are now supported:

Microsoft SQL Server 2022
Oracle 21c
PostgreSQL 15.x
MongoDB 6
DB2 13.1 (when working with IBM DBt for z/OS)
IBM DB2 for iSeries 7.5

Target endpoint versions

The following target endpoint versions are now supported:

Microsoft SQL Server 2022
PostgreSQL 15.x
Databricks (Cloud Storage): Databricks 12.2 LTS and Databricks SQL Serverless
Databricks Lakehouse (Delta): Databricks 12.2 LTS and Databricks SQL Serverless

Resolved issues

Resolved issues in May 2023 service release 1

This section lists the issues resolved since Replicate May 2023 initial release.

Jira issue: RECOB-7251

Salesforce case: 87864

Type: Issue

Component/Process: Kafka Target

Description: Tasks configured to use the Confluent Schema Registry would fail when topic schema validation used a subject name strategy that differed from the topic. The issue was resolved using an internal parameter.

Jira issue: RECOB-7386

Salesforce case: N/A

Type: Issue

Component/Process: ODBC with CDC Source

Description: Extracting data would sometimes lock the source tables. The issue was resolved using an internal parameter.

Jira issue: RECOB-6976

Salesforce case: 63758

Type: Enhancement

Component/Process: SAP HANA Source (Trigger CTS mode)

Description: Added support for a configurable sleep interval after an empty Trigger Table session.

Jira issue: RECOB-7390

Salesforce case: 83518

Type: Issue

Component/Process: SAP Extractor Source

Description: When Extractor unload job failed or was canceled on the SAP side, the extractor status would be changed to Completed instead of Suspended.

Jira issue: RECOB-7323

Salesforce case: 85012

Type: Issue

Component/Process: SAP Extractor Source

Description: When a specific Extractor Full Load was aborted, the next Full Load would replicate the data retrieved by the aborted Extractor Full Load.

Jira issue: RECOB-7128

Salesforce case: N/A

Type: Enhancement

Component/Process: Amazon Redshift Target, Amazon S3 Target

Description: Added support for IAM Roles Anywhere authentication under a feature flag.

Jira issue: RECOB-7321

Salesforce case: N/A

Type: Issue

Component/Process: Replicate for HDD

Description: The RepAgent tag was changed from QlikReplicate to Qlik_QlikCloudDataIntegration.

Jira issue: RECOB-7355

Salesforce case: 67689

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader, Replicate on Linux

Description: The task would fail when a wide Oracle table contained unused or unsupported columns, or LOB columns that were not replicated.

Jira issue: RECOB-7182

Salesforce case: 86992

Type: Issue

Component/Process: Microsoft SQL server Source

Description: The sp_repldone truncation prevention option would not work with a specific collation.

Jira issue: RECOB-7348

Salesforce case: 90678

Type: Issue

Component/Process: SAP Application (DB)

Description: SAP Pool table numeric values less than 0.1 would be replicated incorrectly.

Jira issue: RECOB-7209

Salesforce case: N/A

Type: Enhancement

Component/Process: Engine

Description: Added support for QCS running Replicate tasks without a license.

Jira issue: RECOB-7334

Salesforce case: 84894

Type: Issue

Component/Process: SAP ODP Source

Description: Scheduler log messages would show an incorrect data set name and wrongly formatted time.

Jira issue: RECOB-7311

Salesforce case: N/A

Type: Enhancement

Component/Process: IBM DB2 for z/OS Target

Description: Performance was improved when using the zload utility to load files.

Jira issue: RECOB-7262

Salesforce case: 89358

Type: Issue

Component/Process: SAP ODP Source

Description: Data conversion problems would result in task failure or incorrect data replication.

Jira issue: RECOB-7276, RECOB-7358, RECOB-7373

Salesforce case: N/A

Type: Enhancement

Component/Process: Confluent Cloud Target

Description: New Confluent Cloud target endpoint.

Jira issue: RECOB-7158, RECOB-6925

Salesforce case: 77211

Type: Issue

Component/Process: SAP Extractor Source

Description: This resolves the following issues:

DELTA per table would only start after all Full Loads completed
Reloading a table would prevent all DELTAs
An error in a table during Full Load would prevent DELTAs from scheduling

Jira issue: RECOB-7308

Salesforce case: 78261

Type: Issue

Component/Process: SAP HANA Source (Trigger mode)

Description: After applying a transport for maintenance to ACDOCA all CDC INSERT and UPDATE activity would be replicated as DELETEs.

Jira issue: RECOB-7289

Salesforce case: 80830

Type: Issue

Component/Process: Kafka Target or Change Tables

Description: The "DELETE and INSERT when updating a primary key column" Change Processing Tuning option would not work when a table contained LOB columns.

Jira issue: RECOB-7228

Salesforce case: 84728

Type: Issue

Component/Process: Transformation

Description: It would not be possible to execute a transformation expression that separated keywords with a new line.

Jira issue: RECOB-7149

Salesforce case: 87050

Type: Issue

Component/Process: ODBC Source

Description: Replicate Server would crash when opening the table settings of the ODBC source endpoint.

Jira issue: RECOB-7106

Salesforce case: 60993

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: Replicate would sometimes take an excessively long time to read the online transaction log. An internal property was added to control the number of rows being read in one request.

Jira issue: RECOB-7076

Salesforce case: N/A

Type: Issue

Component/Process: MySQL Source

Description: When a NOT NULL column was defined as AUTOINCREMENTED, it would be exposed as NULLABLE.

Jira issue: RECOB-7146

Salesforce case: N/A

Type: Issue

Component/Process: IBM DB2 for z/OS Source

Description: The task log would display the following incorrect warning when replicating from IBM DB2 on z/OS 13.1 (which is certified):

Replicate has not been certified to work with DB2 z/OS version '13.01.0005', and therefore any issues you may encounter with this version are not covered by your license agreement.

Jira issue: RECOB-7077

Salesforce case: N/A

Type: Enhancement

Component/Process: Engine

Description: A "The task stopped abnormally” event with a FATAL/RECOVERABLE indication was added to the task audit.

Jira issue: RECOB-7041

Salesforce case: 71489

Type: Issue

Component/Process: Engine

Description: RAW data type values which were exposed as a 32 character hex would be truncated to 16 characters.

Jira issue: RECOB-6970

Salesforce case: 80054

Type: Issue

Component/Process: Teradata Source

Description: When a table included a VARCHAR column, the query syntax in Parallel Load using segments would be incorrect.

Jira issue: RECOB-7079

Salesforce case: 82825

Type: Issue

Component/Process: Microsoft Azure SQL (MS-CDC) and Microsoft SQL Server (MS-CDC) targets

Description: The task would fail when DB_ID() returned a different value than database_id in sys.databases. The issue was resolved using an internal parameter.

Jira issue: RECOB-7165

Salesforce case: N/A

Type: Issue

Component/Process: Log Stream

Description: The Log Stream Replication task would stop with an error if Replicate LOB columns was disabled in the settings of the associated Log Stream task.

Jira issue: RECOB-7110

Salesforce case: 82493

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: When a column was split between slots, UPDATE operations would sometimes crash target components.

Jira issue: RECOB-7150

Salesforce case: 80805

Type: Issue

Component/Process: UI Monitor

Description: When scrolling to the right of a log file in the Log File Viewer, the log file would auto-scroll up a few rows.

Jira issue: RECOB-6951

Salesforce case: 78106

Type: Issue

Component/Process: Microsoft Azure ADLS Target

Description: When the If target table already exists Full Load settings option was set to Do Nothing, Replicate would override Parquet and compressed files instead of creating new files with a different name.

Jira issue: RECOB-7096

Salesforce case: 72438

Type: Issue

Component/Process: Salesforce (Incremental Load) Source

Description: Changes would not be captured from a table that was suspended and then reloaded.

Jira issue: RECOB-7114

Salesforce case: 84173

Type: Issue

Component/Process: IBM DB2 for z/OS Source

Description: The R4Z DO4GRANT JCL was updated to grant the R4Z user permissions on the SYSIBM.SYSTABLEPART and SYSIBM.SYSPARMS tables.

Jira issue: RECOB-6998

Salesforce case: 80582

Type: Issue

Component/Process: MySQL Source

Description: When replicating a high volume of data, starting the task or retrieving the table list would take an excessively long time or get stuck. The issue was resolved using an internal parameter.

Jira issue: RECOB-7082

Salesforce case: 77977

Type: Issue

Component/Process: SAP Application (DB) Source

Description: Trailing key fields that were not set in the pool table would not be replicated correctly.

Jira issue: RECOB-6992

Salesforce case: 56522

Type: Enhancement

Component/Process: Oracle Source - Replicate Log Reader

Description: Added support for Oracle DIRECT INSERT statements with PARALLEL_DML for non-compressed pages.

Resolved issues in Replicate May 2023 initial release

This section lists the issues resolved since Replicate November 2022 SR1.

Jira issue: RECOB-6286

Salesforce case: 58375

Type: Issue

Component/Process: Teradata Target

Description: When data errors were encountered during Full Load, the task would continue and the affected table would appear as completed successfully in the Replicate monitor. Now, the task will fail and appropriate error will be shown, together with the existing warning message.

Jira issue: RECOB-6889

Salesforce case: N/A

Type: Issue

Component/Process: SAP Extractor Source

Description: When removing tables from a task, an "out of bound" error would occur.

Jira issue: RECOB-6886

Salesforce case: N/A

Type: Issue

Component/Process: SAP Application (DB)

Description: When Replicate was installed on Linux, no data would be captured from cluster and transparent tables.

Jira issue: RECOB-6865

Salesforce case: 77243

Type: Issue

Component/Process: SAP ODP Source

Description: Unload would fail when the projection list did not include all ODP object columns.

Jira issue: RECOB-6847

Salesforce case: 72990

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: The task would fail when starting from timestamp in "Tlog Only Mode" when the datetime format was dmy.

Jira issue: RECOB-6788

Salesforce case: 72273

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: The table would sometimes be suspended with an "Ignore Split Redo" warning message, when using the compatible Oracle version (12.1).

Jira issue: RECOB-6767

Salesforce case: 73287

Type: Issue

Component/Process: SAP Extractor Source

Description: Tasks would sometimes fail with a "table not found" error.

Jira issue: RECOB-6764

Salesforce case: N/A

Type: Issue

Component/Process: SAP ODP Source

Description: The first "Capture" delta request to SAP would initiate a delta init job, which would remove SAP definitions defined during the previous "LOAD" job.

Jira issue: RECOB-6749

Salesforce case: 73990

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: Added an internal parameter for backup folder preprocessing control.

Jira issue: RECOB-6689

Salesforce case: 56864

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: Data would sometimes be missing when performing Direct Insert into compressed tables.

Jira issue: RECOB-6683

Salesforce case: 56864

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: Data would sometimes be missing when performing Direct Insert into HCC tables.

Jira issue: RECOB-6748

Salesforce case: N/A

Type: Enhancement

Component/Process: Snowflake on Amazon, Snowflake on Azure, Snowflake on Google

Description: The Snowflake partner application name will now be set when an ODBC connection is established.

Jira issue: RECOB-6733

Salesforce case: 72166

Type: Issue

Component/Process: SAP Application (DB)

Description: SAP nullable columns in transparent tables would be exposed as NOT NULL.

Jira issue: RECOB-6712

Salesforce case: 68171

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: Binary data type columns with 0s would appear as null on a PostgreSQL target.

Jira issue: RECOB-6651

Salesforce case: N/A

Type: Enhancement

Component/Process: PostgreSQL Target

Description: It is now possible via an internal parameter to instruct Replicate to ignore extra columns when loading data to the target.

Jira issue: RECOB-6702

Salesforce case: N/A

Type: Issue

Component/Process: SAP ODP

Description: Data capture would sometimes fail with a "Table not found" error.

Jira issue: RECOB-6705

Salesforce case: 68612

Type: Issue

Component/Process: IBM DB2 for z/OS Source

Description: The task would crash when the DB2 version was not certified.

Jira issue: RECOB-6631

Salesforce case: 56093

Type: Issue

Component/Process: MongoDB Source

Description: Replication of MongoDB DATE and LONG data types changed format after upgrading the driver.

Jira issue: RECOB-6653

Salesforce case: 64079

Type: Issue

Component/Process: MongoDB Source

Description: When using binary type 3 as _id, the "_id" would be different than on the MongoDB source.

Jira issue: RECOB-6648

Salesforce case: 56864

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: MULTI and DIRECT INSERT operations into HCC tables would be ignored with the following warning message:

because of unknown row piece '02"'

Jira issue: RECOB-6612

Salesforce case: 66248

Type: Issue

Component/Process: Google BigQuery Target

Description: When using the keyword LOOKUP for a schema name, the task would fail.

Jira issue: RECOB-6625

Salesforce case: 55345

Type: Issue

Component/Process: Kafka Target

Description: When performing Test Connection on Linux, excessive memory would be consumed and not released.

Jira issue: RECOB-6623

Salesforce case: 68171

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: When replicating binary data type columns, trailing zeros would be missing.

Jira issue: RECOB-6598

Salesforce case: N/A

Type: Issue

Component/Process: SAP ODP Source

Description: SAP server errors and exceptions would not be reported by Replicate.

Jira issue: RECOB-6608

Salesforce case: N/A

Type: Issue

Component/Process: SAP ODP Source

Description: Resuming a task would fail with a stream position error when the ODP node description contained special characters.

Jira issue: RECOB-6617

Salesforce case: 64536

Type: Issue

Component/Process: Sorter

Description: When a task was configured with SAP Application (DB), reloading a table would sometimes cause duplicate keys if the table was reloaded while changes were being applied to it.

Jira issue: RECOB-6618

Salesforce case: 67786

Type: Issue

Component/Process: SAP HANA Source (Log Based)

Description: When the ROWID key value was absent, DELETE operations would sometimes fail on the target and INSERT operations would sometimes be missed.

Jira issue: RECOB-6626

Salesforce case: N/A

Type: Issue

Component/Process: Databricks Target

Description: The proxy storage checkbox would not be enabled after upgrading from 2022.5 (although it was enabled before the upgrade).

Jira issue: RECOB-6604

Salesforce case: 65993

Type: Issue

Component/Process: Snowflake Azure Target

Description: When specifying an Azure blob staging folder without a slash ('/'), Replicate would create the folder without its first letter.

Jira issue: RECOB-6581

Salesforce case: 66709

Type: Issue

Component/Process: SAP HANA Source (Log Based)

Description: Tables would sometimes be suspended with the following warning message:

Failed to get data values for ... data is set to NULL

Jira issue: RECOB-6547

Salesforce case: N/A

Type: Issue

Component/Process: Databricks (Cloud Storage)

Description: When the Optimize File Uploads tasks setting was enabled together with the endpoint's compression option, temporary files would not be deleted from the Replicate machine (which would impact disk space).

Jira issue: RECOB-6370

Salesforce case: 57757

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: When Oracle was restarted with the RESETLOGS command, the task would fail.

Jira issue: RECOB-6572

Salesforce case: 60522

Type: Issue

Component/Process: Transformations

Description: Added logging to troubleshoot transformation issues.

Jira issue: RECOB-6567

Salesforce case: 66516

Type: Issue

Component/Process: SAP HANA Source (Log Based)

Description: Tables would sometimes be suspended without any log message.

Jira issue: RECOB-6532

Salesforce case: 63213

Type: Issue

Component/Process: Databricks Lakehouse (Delta)

Description: When the Apply batched changes to multiple tables concurrently option was enabled, the task would occasionally fail to apply the changes but would eventually succeed after retries.

Jira issue: RECOB-6925

Salesforce case: 77211

Type: Issue

Component/Process: SAP Extractor Source

Description: The extractors would sometimes not schedule the delta jobs after Full Load completes.

Jira issue: RECOB-6908

Salesforce case: 77577

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: When performing an UPDATE on an IOT table, a column value would sometimes be missing.

Jira issue: RECOB-6532

Salesforce case: 63213

Type: Issue

Component/Process: Databricks Lakehouse (Delta)

Description: If the user was set as British, then when starting from timestamp with the day of the month higher than 12, the following error would sometimes occur:

The conversion of a varchar data type to a datetime data type resulted in an out-of-range value

Known issues

Jira issue: RECOB-7040

Salesforce case: 71489

Type: Issue

Component/Process: SAP Extractor Source

Description: RAW data type values which are exposed as 32 character hex, are truncated to 16 characters.

Jira issue: RECOB-6997

Salesforce case: 80582

Type: Issue

Component/Process: MySQL Source

Description: When replicating a high volume of data, Replicate either fails to start the task/retrieve the table list, or takes an excessive amount of time.

Jira issue: RECOB-6863

Salesforce case: 56522

Type: Issue

Component/Process: Oracle Source - Replicate Log Reader

Description: Missing support for Oracle DIRECT INSERT statements with PARALLEL_DML for non-compressed pages.

Jira issue: RECOB-6969

Salesforce case: 80054

Type: Issue

Component/Process: Teradata Source, Parallel Load

Description: The query syntax in Parallel Load from a Teradata Source using segments is incorrect when one of the columns is VARCHAR.

Jira issue: RECOB-6950

Salesforce case: 78106

Type: Issue

Component/Process: Microsoft Azure ADLS Target

Description: When the Full Load If target table already exists option is set to Do Nothing, Parquet and compressed files are overwritten instead of being created with a different name.

Jira issue: RECOB-6924

Salesforce case: 77211

Type: Issue

Component/Process: SAP Extractor Source

Description: The extractors sometimes do not schedule the delta jobs after Full Load completes.

Jira issue: RECOB-6848

Salesforce case: 72990

Type: Issue

Component/Process: Microsoft SQL Server Source

Description: Replicate fails to start from timestamp when the endpoint is configured to capture changes from transaction logs only and the datetime format is DMY.

Downloads

Download this release from the Product Downloads page on Qlik Community.

About Qlik

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.

qlik.com

Qlik Replicate May 2023 Release Notes - Initial Release until Service Release 1