Qlik Data Integration Newsletter - August 2023

[Sonja_Bauernfeind]

Welcome to August's Qlik Data Integration newsletter. Each month, we cover one endpoint and share our top resources, best practices, release updates and upcoming webinars.

Subscribe to the Qlik Data Integration topic to be notified of future editions!  

Index

• Resource Highlights
• Reminder! Qlik Replicate Endpoint Retirement
• Knowledgebase Article
• Endpoint Spotlight: Databricks
• New Issues Reported
• Tips!
• Release Updates
• Fixes
• Upcoming Enhancements
• Troubleshooting tips
• Analyze Qlik Replicate Logs
• Qlik Data Integration Product End of Life Versions

 

Resource Highlights

Reminder! Qlik Replicate Endpoint Retirement

As of July 31st, 2023, the following endpoints have been retired:

Source Endpoints:

• HP Non-Stop Source
• Open VMS RMS Source
• Hadoop Source

Target Endpoints:

• MapR Target
• MapR Streams Target
• MS APS PDW Target
• SAP Sybase IQ Target
• HP Vertica Target
• Actian Vector Target
• Netezza Target
• Pivotal Greenplum Target

See Retirement for Specific Qlik Replicate Endpoints for details.

 

Knowledgebase Article

Endpoint Spotlight: Databricks

Find our latest knowledge base articles for Databrick endpoints. 

New Issues Reported

• Error SSL_connect: wrong version number when using Databricks Lakehouse (Delta) as target
  • Find the resolution in Qlik Replicate with Databricks Lakehouse (Delta) as target: Error SSL_connect: wrong version number.
• Performing operations in transactions apply mode is not supported with Databricks Lakehouse (Delta) as target
  • The Apply Changes replication mode supports tables with a Primary Key only. For solutions see Qlik Replicate with Databricks Lakehouse (Delta) as target: Performing operations in transactions ap....
• Table is failing on full load when loading into Microsoft Azure Databricks Delta Target
  • Find the resolution in Qlik Replicate: Table is failing on full load when loading into Microsoft Azure Databricks Delta Tar....
• Qlik Replicate Server crashes when browsing databases in Microsoft Azure Databricks Target 
  • Clicking Browse in the database endpoint settings of a Microsoft Azure Databricks target crashes the Qlik Replicate server. Find the resolution in Qlik Replicate Server crashes when browsing databases in Microsoft Azure Databricks Target.

Tips!

• Qlik Replicate and Databrick Unity Catalog feature flag support
  • Qlik Replicate added support for Databricks Unity Catalog using feature flag from 2022.11 SP02. See Qlik Replicate added support for Databricks Unity Catalog using feature flag from 2022.11 SP02 for details.
• Qlik Replicate: Databricks Operations per second is over the account limit
  
  • Qlik Replicate tasks may fail with Operations per second is over the account limit errors. Find the resolution in Qlik Replicate: Databricks Operations per second is over the account limit.
• Best Practice: Qlik Replicate: Databricks Delta Wstring data type truncation
  • Starting with Databricks Delta Driver version to 8.x and above, Varchar data type will be used for strings. Varchar data type is no longer limited by this character size so no truncation will occur when Wstring data type is used with limits higher than 255 characters.

Release Updates

Fixes

Qlik Replicate May 2023 patches

Component/Process: Databricks (Cloud Storage)

Description: When reconnecting after recoverable error on uploading file, the last file does not get uploaded, resulting in missing data.

Component/Process: Databricks Lakehouse (Delta)

Description: When using merge, if one of the columns in a Unique Index was NULL the changes would not be applied correctly. The issue was resolved using a Feature Flag at task and server level.

Upcoming Enhancements 

Qlik Replicate August 2023 IR

• New Confluent Endpoint (based on Kafka) 
• S3 authentication support with AWS IAM Roles Anywhere on S3/Redshift endpoints 
• Ability to Set Isolation Level on ODBC with CDC (Ability to Set Isolation Level on ODBC with CDC) 

Qlik Replicate November 2023 IR

• Expose stream buffer tuning parameters in task settings (Replicate & QEM - Expose stream buffer tuning parameters in task settings )
• Confluent Schema Register using BOTH authentication methods (Public Key + Username & password)
• Snowflake - Support Transactions in Upsert Mode
• Boolean Data Type support for PG sources and Bit support for SQL Server targets
• Google BigQuery - Secure current authentication (no longer have credential files on customer file system)
• Aurora PostgreSQL source - without the "superuser" role certification
• Azure Database for MySQL Flexible Server - Source & Target Certification
• MariaDB 10.6 to 10.11 versions on MySQL - certification
• MariaDB 10.6 version on Amazon RDS - certification

Troubleshooting tips

Analyze Qlik Replicate Logs 

An evergreen pair of articles helps you read and analyze Qlik Replicate log files:

How to analyze a Qlik Replicate log
List of the error types in Qlik Replicate

•  

Qlik Data Integration Product End of Life Versions 

Qlik Release	Qlik Replicate / Enterprise Manager	End of Support Date	Qlik Compose	End of Support Date
February 2021	November 2020 SR1	November 2022	February 2021	February 2023
May 2021	May 2021	May 2023	May 2021	May 2023
August 2021	May 2021 SR1	May 2023	August 2021	August 2023
November 2021	November 2021	November 2023	November 2021	November 2023

For more information, see Qlik Product Lifecycles.