Qlik Compose December 2024 Initial Release until Service Release 1

[Charlton_Book]

Table of Contents

• What's new
• Migration and upgrade
• Upgrade paths
• Licensing
• Migrating from Compose for Data Lakes
• Compatibility with related Qlik products
• End of Life/Support and deprecated features
• Resolved issues
• Service Release 1: Issues resolved since Qlik Compose December 2024 Initial Release
• Initial Release: Issues resolved in Qlik Compose December 2024 Initial Release
• Known issues
• Downloads

What's new

Learn about new features and improvements in Qlik Compose.

Migration and upgrade

This section describes the supported upgrade and migration paths.

Upgrade paths

Direct upgrade is supported from Compose November 2023 (and its service releases) only. Customers wishing to upgrade from earlier Compose versions should first upgrade to Compose November 2023 and then to Compose December 2024.

Licensing

Existing Compose for Data Warehouses customers who want to create and manage Data Warehouse projects only in Qlik Compose can use their existing license. Similarly, existing Compose for Data Lakes customers who want to create and manage Data Lake projects only in Qlik Compose can use their existing license.

Customers migrating from Qlik Compose for Data Warehouses or Qlik Compose for Data Lakes, and who want to create and manage both Data Warehouse projects and Data Lakes projects in Qlik Compose, will need to obtain a new license. Customers upgrading from Compose February 2021 can continue using their existing license.

It should be noted that the license is enforced only when trying to generate, run, or schedule a task (via the UI or API ). Other operations such as Test Connection may also fail if you do not have an appropriate license.

Migrating from Compose for Data Lakes

Compose for Data Lakes has been superseded by Qlik Compose. Existing Compose for Data Lakes customers can migrate their projects from Qlik Compose for Data Lakes to Qlik Compose. You can migrate both your project definitions and your data although the latter is only required if you need to migrate production data.

For migration instructions, see the Qlik Compose August 2021 Release notes.

Note: Migration can be performed from Compose for Data Lakes 6.6 only.

Compatibility with related Qlik products

Qlik Replicate is required for landing data into the data warehouse or storage while Qlik Enterprise Manager allows you to monitor and control Compose tasks running on different servers. This section lists the supported versions for each of these products.

Compose December 2024 is compatible with the following Replicate and Enterprise Manager versions:

• Qlik Replicate - Qlik Replicate November 2024, Qlik Replicate May 2024, Qlik Replicate November 2023, and Qlik Replicate May 2023 latest service release.
• Enterprise Manager - Qlik Enterprise Manager November 2024, Qlik Enterprise Manager May 2024, and Qlik Enterprise Manager November 2023.

End of Life/Support and deprecated features

The following versions are no longer supported:

• Databricks 9.1 LTS

Resolved issues

Service Release 1: Issues resolved since Qlik Compose December 2024 Initial Release

The following issues have been resolved since Qlik Compose December 2024 initial release:

Jira issue: RECOB-9113 

Salesforce case: 322990 

Type: Issue 

Component/Process: Server 

Description: During Full Load, in rare scenarios, the wrong Replicate task status would be shown in the Compose monitor. 
 

Jira issue: RECOB-9257 

Salesforce case: 335597 

Type: Issue 

Component/Process: Data Mart Validation 

Description: During data mart validation, when the fact table contained an OID column belonging to a deleted relation, the following error would be shown: 

SYS,GENERAL_EXCEPTION, Object reference not set to an instance of an object 

Jira issue: RECOB-9327 

Salesforce case: 345805 

Type: Issue 

Component/Process: Data Mart - Snowflake on AWS 

Description: When running a data mart task, the following error would occur: 

sqlstate 42000, errorcode 904, message SQL compilation error 

Jira issue: RECOB-9508 

Salesforce case: 355592 

Type: Issue 

Component/Process: ETL Statements 

Description: When a filter and a data quality cleansing rule were defined on the same column, the data warehouse task would fail with the following error (excerpt): 

Incorrect syntax near the keyword 

Initial Release: Issues resolved in Qlik Compose December 2024 Initial Release

The following issues have been resolved in Qlik Compose December 2024 Initial Release:

Jira issue: RECOB-8771 

Salesforce case: 173509 

Type: Issue 

Component/Process: Workflows 

Description: Validation checks for complex workflow would be excessively long before running the first operation. 

Jira issue: RECOB-8899 

Salesforce case: 303295 

Type: Issue 

Component/Process: Data Lake Project – ETL Statement 

Description: If a record was updated in a partition that was still open after the CDC task started, the record would sometimes not be archived. 

Jira issue: RECOB-8885 

Salesforce case: 310095 + 310451 

Type: Issue 

Component/Process: Data Marts (SQL Server Target) 

Description: Data mart generation would sometimes fail with a “Duplicate columns detected” error. 

Jira issue: RECOB-8922  

Salesforce case:  309076 

Type: Issue 

Component/Process: Data Marts 

Description: An incorrect fact INSERT statement would be generated if the fact contained type 2 entities and a transaction date. 

Jira issue: RECOB-8808  

Salesforce case: 297099 

Type: Issue 

Component/Process: ETL Statements 

Description: When two instances of the same task ran in parallel, one of the tasks would fail as it would try to access a temp table which was deleted by the other task. 

Jira issue: RECOB-9032 

Salesforce case: 317354 

Type: Issue 

Component/Process: Amazon Redshift, ETL Statements 

Description:  Due to a driver limitation, when inserting more than 2 billion records, the data warehouse task would sometimes fail with the following error: 

Parameter MappingCount_E12309_M001 has no value 

The issue was resolved using the latest JDBC driver: redshift-jdbc42-2.1.0.30 

Jira issue: RECOB- 9031 

Salesforce case: 317518 

Type: Issue 

Component/Process: Microsoft SQL Server Target 

Description: Using a Transactional Fact with Type 1 dimensions that contained a filter would result in an incorrect statement being generated. 

Jira issue: RECOB-8991 

Salesforce case:  308524 

Type: Issue 

Component/Process: Snowflake Target 

Description: When using key pair authentication, the key would break after deployment. 

Jira issue: RECOB-8727 

Salesforce case: 296636 

Type: Issue 

Component/Process: Project documentation 

Description: When trying to generate project documentation, the following error would sometimes occur:  

The given key was not found in the dictionary 

Jira issue: RECOB-8776 

Salesforce case: 165516 

Type: Issue 

Component/Process: Security vulnerability 

Description: The index.html and demo.html files were removed from the Compose installation to address vulnerability CVE-2019-10768. 

Jira issue: RECOB-8567 

Salesforce case: 171604 

Type: Issue 

Component/Process: Data Marts 

Description: Data mart generation would fail when the Date dimension was created manually. 

Jira issue: RECOB-8569 

Salesforce case: 166504 

Type: Issue 

Component/Process: Workflows 

Description: When a workflow with a parallel split failed, the monitor would still show it as running. 

Jira issue: RECOB-8622 

Salesforce case: 170800 

Type: Issue 

Component/Process: Logging  

Description: Not all files were deleted from <INSTALL-DIR>\java\data\projects\<project-name>\logs, even though the log retention policy was configured to delete them. 

Jira issue: RECOB-8639 

Salesforce case: 173669 

Type: Issue 

Component/Process: Data Marts  

Description: During the initial run of data mart tasks with transactional facts, Compose would try to update the table instead of just inserting the data. 

Jira issue: RECOB-8197 

Salesforce case: 139123 

Type: Issue 

Component/Process: UI - Monitor  

Description: The task would be shown as completed successfully (in the monitor and in the log), but the progress bar would show that some of the ETL statements had still not been completed.

Jira issue: RECOB-8673 

Salesforce case: 292717 

Type: Issue 

Component/Process: Data Marts, ETL Statements  

Description: When attempting to augment an aggregated fact with a dimension through the 'Add to Star Schema' option, the system would sometimes generate the following error: 

Invalid column name _OID 

Jira issue: RECOB-8690 

Salesforce case: 172614 

Type: Issue 

Component/Process: Data Warehouse CDC Task

Description: Generating a data warehouse CDC task would take a long time when a project was configured to use the same database (for landing and archiving, for example), but with a different schema. 

Jira issue: RECOB-8383 

Salesforce case: 161351 

Type: Enhancement 

Component/Process: UI Monitor 

Description: Complex workflow validation will now be skipped to improve monitor performance. 

Jira issue: RECOB-8472 

Salesforce case: 164789 

Type: Issue 

Component/Process: CLI 

Description: Data mart tables would be dropped while executing the following command: 

ComposeCli.exe generate_project 

Jira issue: RECOB-8462 

Salesforce case: 165355 

Type: Issue 

Component/Process: Cloudera Data Platform (CDP) 

Description: When there were multiple records with the same ID and From Date, Compose would not pick up the latest Change Table record. 

Jira issue: RECOB-8494 

Salesforce case: 167502 

Type: Issue 

Component/Process: UI 

Description: Task search would not work as expected. 

Jira issue: RECOB-8517 

Salesforce case: Qlik Internal 

Type: Issue 

Component/Process: UI 

Description: When a custom ETL contained special characters, the export_csv command would fail. 

Jira issue: RECOB-8287 

Salesforce case: 154357

Type: Issue

Component/Process: Metadata Manager

Description: Checking the metadata when adjusting the data mart would sometimes be excessively slow.

Jira issue: RECOB-8333 

Salesforce case: 158494

Type: Issue

Component/Process: CLI

Description: The ComposeCli.exe generate_project command would complete without any errors when the data warehouse was not synced with the model.

Jira issue: RECOB-7821 

Salesforce case: 151213

Type: Issue

Component/Process: Security 

Description: Google Guava was upgraded from 31.0.1 to 33.1.0, which fixes vulnerability CVE-2023-2976.

Jira issue: RECOB-8074

Salesforce case: 151213

Type: Issue

Component/Process: Security

Description: Removed the postgresql-42.6.0.jar from Compose to address vulnerability CVE-2024-1597.

Jira issue: RECOB-7860

Salesforce case: 151213

Type: Issue

Component/Process: Security

Description: The org.eclipse.jetty:jetty-http web server component was upgraded to a version that fixes the following vulnerabilities: CVE-2023-36478, CVE-2023-36479, and CVE-2023-40167

Jira issue: RECOB-8319

Salesforce case: 158765

Type: Issue

Component/Process: Data Marts

Description: Retroactive changes would create too many obsolete records in the dimension tables.

Jira issue: RECOB-8378

Salesforce case: 154292

Type: Issue

Component/Process: Connectivity, Microsoft Azure Managed SQL Server 

Description: Data warehouse tasks would sometimes fail to complete.

Jira issue: RECOB-8096 

Salesforce case: 147298

Type: Issue

Component/Process: SQL Server

Description: Compose would fail to connect to SQL Server when using Windows Authentication.

Jira issue: RECOB-8154 

Salesforce case: 129455

Type: Issue

Component/Process: Logging

Description: Debugging messages related to connection issues would fill up the data mart task log and severely impact disk space.

Jira issue: RECOB-8202 

Salesforce case: 129455

Type: Issue

Component/Process: Data Marts

Description: When two different dimensions used the same data warehouse table with self-reference tables, data mart generation would fail with a "duplicate columns detected" error. 

Jira issue: RECOB-8162

Salesforce case: 151211

Type: Issue

Component/Process: Metadata Manager, Google BigQuery

Description: When using the NUMERIC data type with non-default size and precision, the scale and precision would be incorrectly converted.

Jira issue: RECOB-8100

Salesforce case: 142698

Type: Issue

Component/Process: Databricks Delta

Description: The Databricks SQL Warehouse ETL statement for CDC would sometimes fail with a casting error when querying the attrep_cdc_partitions control table.

Jira issue: RECOB-8233

Salesforce case: Qlik Internal

Type: Issue

Component/Process: Data Marts, ETL Statements

Description: In rare scenarios, the data mart ETL would fail with an "Incorrect syntax error near COALESCE" error. 

Jira issue:  RECOB-7871 

Salesforce case: 126374

Type: Issue

Component/Process: Data Marts, Compose Log File

Description: A redundant "Data mart ID does not exist" log message would be repeatedly written to the Compose log file.

Jira issue:  RECOB-7948 

Salesforce case: 133345

Type: Issue

Component/Process: Data Marts

Description: Multiple column support has been added to the Synapse HASH distribution key. This resolves the following error: "SYS,GENERAL_EXCEPTION, An item with the same key has already been added"

Jira issue: RECOB-7211

Salesforce case: 87096

Type: Enhancement

Component/Process: UI Monitor - Data Lake Projects

Description: Added the ability to sort columns in the Manage Data Storage Tasks and Monitor Details windows.

Jira issue: RECOB-8011

Salesforce case: 139471

Type: Issue

Component/Process: Task Commands   

Description:  A "Failed to start task" error would sometimes be encountered after upgrading.

Jira issue: RECOB-8004

Salesforce case: 129455

Type: Enhancement

Component/Process: Connectivity

Description: To optimize connection recovery, a retry functionality was added when writing to Control tables.

Jira issue:  RECOB-8006

Salesforce case: 139471

Type: Issue

Component/Process: Data Marts, ETL Statements

Description: When generating a data mart task, a "Duplicate columns detected" error would sometimes be encountered due to incorrect handling of multi-path attributes.

Jira issue: RECOB-8020

Salesforce case: 137863

Type: Issue

Component/Process: Data Marts

Description: An incorrect "Attribute used in expression does not exist in the model" error would sometimes be encountered during data mart validation.

Note: The fix uses a "max_transactional_date_tree_depth" parameter to determine the depth of the dimension tree. As this requires a change in the project's repository, please contact Qlik Support for assistance If you think you might have encountered this issue.

Jira issue: RECOB-8022

Salesforce case: 140252

Type: Issue

Component/Process: ETL Statements, Performance, SQL Server

Description: The hashing process for CDC would sometimes consume excessive memory and significantly lengthen task completion time. To benefit from this fix, you need to regenerate the task ETLs after installing the patch.

Jira issue: RECOB-7859

Salesforce case: 127377

Type: Issue

Component/Process: Data Warehouse Adjust 

Description: When adjusting the data warehouse after a change in the model, the following error would sometimes occur:

SYS,GENERAL_EXCEPTION, Sequence contains no elements

Jira issue: RECOB-7803

Salesforce case: 104948

Type: Issue

Component/Process: Data Mart Performance on Snowflake

Description: Data mart performance issues would be encountered after upgrading. This version reverts the changes to the code that caused the performance issues.

Jira issue:  RECOB-7840

Salesforce case:  121509

Type: Issue

Component/Process: ETL Statements, Source Consolidation

Description: ETL generation would sometimes fail when Consolidate uniform sources was enabled and the source type was View.

Jira issue:  RECOB-7637

Salesforce case:  113806

Type: Issue

Component/Process: ETL Statements

Description: When a long "From date" column name was defined in the project settings, the following error would be encountered when automatically adjusting the data warehouse:

Input string not in correct format

Jira issue: RECOB-7387

Salesforce case: 97736

Type: Issue

Component/Process: Documentation

Description: When a Compose project contained environment variables, the following error would occur when generating the product documentation:

Length cannot be less than zero. Parameter name: length

Jira issue: RECOB-7842

Salesforce case: 127234 

Type: Issue

Component/Process: UI

Description: Discovering tables from a MySQL source would sometimes cause the UI to hang.

Jira issue: RECOB-7927

Salesforce case: 129455

Type: Issue

Component/Process: Database Connections

Description: Connections to the database would not be reestablished when a task was paused and resumed.

Jira issue: RECOB-7951

Salesforce case: 132765

Type: Issue

Component/Process: Data Marts, Upgrade

Description: Upgrading from Compose May 2022 removed all links from facts to dimensions.

Known issues

If you have a Compose task that connects to Databricks Compute Platform using OAuth authentication, and you want to monitor the task in Qlik Enterprise Manager, you will need a patched version of Qlik Enterprise Manager. Without the patch, all Compose tasks will be disabled and a json deserialization error will be displayed. To obtain the patch, contact Qlik Support. Alternatively, you can monitor your tasks directly in Compose.

Downloads

Download this release from the Product Downloads page on Qlik Community.

About Qlik

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.

qlik.com