Qlik Compose - November 2023 Initial Release until Service Release 1

[Charlton_Book]

Table of Contents

• What's new
• Migration and upgrade
• Upgrade paths
• Compose upgrade path
• Compose for Data Warehouses upgrade path
• Upgrading from Compose for Data Warehouses 6.6.1 (September 2020) or 7.0 (November 2020):
• Upgrading from unsupported Compose for Data Warehouses versions
• Compose for Data Lakes upgrade path
• Licensing
• Migrating from Compose for Data Lakes
• Compatibility with related Qlik products
• Newly supported platforms and versions
• End of Life/Support and deprecated features
• Resolved issues
• Issues resolved since Compose November 2023 Initial Release
• Issues resolved since Compose May 2022 Service Release 1
• Downloads

What's new

Learn about new features and improvements in Qlik Compose.

Migration and upgrade

This section describes the issues that you may encounter when upgrading/migrating to the new version.

Upgrade paths

Compose upgrade path

Direct upgrade is supported from Compose May 2022 only. Customers upgrading from earlier Compose versions need to first upgrade to Compose May 2022 and then to Compose November 2023.

Compose for Data Warehouses upgrade path

Compose for Data Warehouses has been superseded by Qlik Compose. Existing Compose for Data Warehouses customers can upgrade to Qlik Compose as described below.

Note: For information on the procedure for upgrading from Compose for Data Warehouses to Compose February 2021, see the Qlik Compose February 2021 release notes

Upgrading from Compose for Data Warehouses 6.6.1 (September 2020) or 7.0 (November 2020):

1. Upgrade to Compose February 2021.
2. Upgrade to Compose May 2021.
3. Upgrade to Compose May 2022.
4. Upgrade to Compose November 2023 SR1.

Upgrading from unsupported Compose for Data Warehouses versions

Customers upgrading from Compose for Data Warehouses 6.5 or 6.6:

1. Upgrade to Compose for Data Warehouses 6.6.1.
2. Upgrade to Compose February 2021.
3. Upgrade to Compose May 2021.
4. Upgrade to Compose May 2022 (including SRs).
5. Upgrade to Compose November 2023 SR1.

Customers upgrading from Compose for Data Warehouses 6.3 or 6.4:

1. Upgrade to Compose for Data Warehouses 6.5.
2. Upgrade to Compose for Data Warehouses 6.6.1.
3. Upgrade to Compose February 2021.
4. Upgrade to Compose May 2021.
5. Upgrade to Compose May 2022 (including SRs).
6. Upgrade to Compose November 2023 SR1.

Customers upgrading from Compose for Data Warehouses 3.1 should contact Qlik Support.

Compose for Data Lakes upgrade path

For information on upgrading from Compose for Data Lakes, see Migrating from Compose for Data Lakes below.

Licensing

Existing Compose for Data Warehouses customers who want to create and manage Data Warehouse projects only in Qlik Compose can use their existing license. Similarly, existing Compose for Data Lakes customers who want to create and manage Data Lake projects only in Qlik Compose can use their existing license.

Customers migrating from Qlik Compose for Data Warehouses or Qlik Compose for Data Lakes, and who want to create and manage both Data Warehouse projects and Data Lakes projects in Qlik Compose, will need to obtain a new license. Customers upgrading from Compose February 2021 can continue using their existing license.

It should be noted that the license is enforced only when trying to generate, run, or schedule a task (via the UI or API ). Other operations such as Test Connection may also fail if you do not have an appropriate license.

Migrating from Compose for Data Lakes

Compose for Data Lakes has been superseded by Qlik Compose. Existing Compose for Data Lakes customers can migrate their projects from Qlik Compose for Data Lakes to Qlik Compose. You can migrate both your project definitions and your data although the latter is only required if you need to migrate production data.

For migration instructions, see the Qlik Compose August 2021 Release notes.

Note: Migration can be performed from Compose for Data Lakes 6.6 only.

Compatibility with related Qlik products

Qlik Replicate is required for landing data into the data warehouse or storage while Qlik Enterprise Manager allows you to monitor and control Compose tasks running on different servers. This section lists the supported versions for each of these products.

Compose November 2023 Service Release 1 is compatible with the following Replicate and Enterprise Manager versions:

• Qlik Replicate - Qlik Replicate May 2024, Qlik Replicate November 2023, and Qlik Replicate May 2023 latest service release.
• Enterprise Manager - Qlik Enterprise Manager May 2024, Qlik Enterprise Manager November 2023.

Newly supported platforms and versions

The following versions are now supported:

• Microsoft SQL Server 2022
  
• Oracle 21c
  
• Databricks 12.2 LTS
• Databricks 14.3 LTS (Requires Compose November 2023 Service Release 1)

End of Life/Support and deprecated features

The following versions are no longer supported:

• Oracle 11 and 12
  
• Windows 2012

Resolved issues

Issues resolved since Compose November 2023 Initial Release

Jira issue: RECOB-8287 

Salesforce case: 154357

Type: Issue

Component/Process: Metadata Manager

Description: Checking the metadata when adjusting the data mart would sometimes be excessively slow.

Jira issue: RECOB-8333 

Salesforce case: 158494

Type: Issue

Component/Process: CLI

Description: The ComposeCli.exe generate_project command would complete without any errors when the data warehouse was not synced with the model.

Jira issue: RECOB-7821 

Salesforce case: 151213

Type: Issue

Component/Process: Security 

Description: Google Guava was upgraded from 31.0.1 to 33.1.0, which fixes vulnerability CVE-2023-2976.

Jira issue: RECOB-8074

Salesforce case: 151213

Type: Issue

Component/Process: Security

Description: Removed the postgresql-42.6.0.jar from Compose to address vulnerability CVE-2024-1597.

Jira issue: RECOB-7860

Salesforce case: 151213

Type: Issue

Component/Process: Security

Description: The org.eclipse.jetty:jetty-http web server component was upgraded to a version that fixes the following vulnerabilities: CVE-2023-36478, CVE-2023-36479, and CVE-2023-40167

Jira issue: RECOB-8319

Salesforce case: 158765

Type: Issue

Component/Process: Data Marts

Description: Retroactive changes would create too many obsolete records in the dimension tables.

Jira issue: RECOB-8378

Salesforce case: 154292

Type: Issue

Component/Process: Connectivity, Microsoft Azure Managed SQL Server 

Description: Data warehouse tasks would sometimes fail to complete.

Jira issue: RECOB-8096 

Salesforce case: 147298

Type: Issue

Component/Process: SQL Server

Description: Compose would fail to connect to SQL Server when using Windows Authentication.

Jira issue: RECOB-8154 

Salesforce case: 129455

Type: Issue

Component/Process: Logging

Description: Debugging messages related to connection issues would fill up the data mart task log and severely impact disk space.

Jira issue: RECOB-8202 

Salesforce case: 129455

Type: Issue

Component/Process: Data Marts

Description: When two different dimensions used the same data warehouse table with self-reference tables, data mart generation would fail with a "duplicate columns detected" error. 

Jira issue: RECOB-8162

Salesforce case: 151211

Type: Issue

Component/Process: Metadata Manager, Google BigQuery

Description: When using the NUMERIC data type with non-default size and precision, the scale and precision would be incorrectly converted.

Jira issue: RECOB-8100

Salesforce case: 142698

Type: Issue

Component/Process: Databricks Delta

Description: The Databricks SQL Warehouse ETL statement for CDC would sometimes fail with a casting error when querying the attrep_cdc_partitions control table.

Jira issue: RECOB-8233

Salesforce case: Qlik Internal

Type: Issue

Component/Process: Data Marts, ETL Statements

Description: In rare scenarios, the data mart ETL would fail with an "Incorrect syntax error near COALESCE" error. 

Jira issue:  RECOB-7871 

Salesforce case: 126374

Type: Issue

Component/Process: Data Marts, Compose Log File

Description: A redundant "Data mart ID does not exist" log message would be repeatedly written to the Compose log file.

Jira issue:  RECOB-7948 

Salesforce case: 133345

Type: Issue

Component/Process: Data Marts

Description: Multiple column support has been added to the Synapse HASH distribution key. This resolves the following error: "SYS,GENERAL_EXCEPTION, An item with the same key has already been added"

Jira issue: RECOB-7211

Salesforce case: 87096

Type: Enhancement

Component/Process: UI Monitor - Data Lake Projects

Description: Added the ability to sort columns in the Manage Data Storage Tasks and Monitor Details windows.

Jira issue: RECOB-8011

Salesforce case: 139471

Type: Issue

Component/Process: Task Commands   

Description:  A "Failed to start task" error would sometimes be encountered after upgrading from PR16 to PR18.

Jira issue: RECOB-8004

Salesforce case: 129455

Type: Issue

Component/Process: Connectivity

Description: To optimize connection recovery, a retry functionality was added when writing to Control tables.

Jira issue:  RECOB-8006

Salesforce case: 139471

Type: Issue

Component/Process: Data Marts, ETL Statements

Description: When generating a data mart task, a "Duplicate columns detected" error would sometimes be encountered due to incorrect handling of multi-path attributes.

Jira issue: RECOB-8020

Salesforce case: 137863

Type: Issue

Component/Process: Data Marts

Description: An incorrect "Attribute used in expression does not exist in the model" error would sometimes be encountered during data mart validation.

Note: The fix uses a "max_transactional_date_tree_depth" parameter to determine the depth of the dimension tree. As this requires a change in the project's repository, please contact Qlik Support for assistance If you think you might have encountered this issue.

Jira issue: RECOB-8022

Salesforce case: 140252

Type: Issue

Component/Process: ETL Statements, Performance, SQL Server

Description: The hashing process for CDC would sometimes consume excessive memory and significantly lengthen task completion time. To benefit from this fix, you need to regenerate the task ETLs after installing the patch.

Jira issue: RECOB-7859

Salesforce case: 127377

Type: Issue

Component/Process: Data Warehouse Adjust 

Description: When adjusting the data warehouse after a change in the model, the following error would sometimes occur:

SYS,GENERAL_EXCEPTION, Sequence contains no elements

Jira issue: RECOB-7803

Salesforce case: 104948

Type: Issue

Component/Process: Data Mart Performance on Snowflake

Description: Data mart performance issues would be encountered after upgrading. This version reverts the changes to the code that caused the performance issues.

Jira issue:  RECOB-7840

Salesforce case:  121509

Type: Issue

Component/Process: ETL Statements, Source Consolidation

Description: ETL generation would sometimes fail when Consolidate uniform sources was enabled and the source type was View.

Jira issue:  RECOB-7637

Salesforce case:  113806

Type: Issue

Component/Process: ETL Statements

Description: When a long "From date" column name was defined in the project settings, the following error would be encountered when automatically adjusting the data warehouse:

Input string not in correct format

Jira issue: RECOB-7387

Salesforce case: 97736

Type: Issue

Component/Process: Documentation

Description: When a Compose project contained environment variables, the following error would occur when generating the product documentation:

Length cannot be less than zero. Parameter name: length

Jira issue: RECOB-7842

Salesforce case: 127234 

Type: Issue

Component/Process: UI

Description: Discovering tables from a MySQL source would sometimes cause the UI to hang.

Jira issue: RECOB-7927

Salesforce case: 129455

Type: Issue

Component/Process: Database Connections

Description: Connections to the database would not be reestablished when a task was paused and resumed.

Jira issue: RECOB-7951

Salesforce case: 132765

Type: Issue

Component/Process: Data Marts, Upgrade

Description: Upgrading from Compose May 2022 to Compose November 2023 removed all links from facts to dimensions.

Issues resolved since Compose May 2022 Service Release 1

Jira issue: RECOB-7474

Salesforce case: 103547

Type: Issue

Component/Process:  Data Marts, ETL Statements

Description: Under very special conditions (a child table having multiple relationships to parent table P1 and having multiple relationships to parent table P2), the generated ETL for handling dimensions would be incorrect.

Jira issue: RECOB-7777

Salesforce case: 118205

Type: Issue
Component/Process: CDC, Azure Synapse Analytics and SQL Server

Description: When capturing changes from NVARCHAR columns, the CDC task would fail when the Windows environment variable "qlk__UseSatelliteRIS" was set to "1".

Jira issue: RECOB-7766

Salesforce case: 115292

Type: Issue

Component/Process: UI Monitor - Performance

Description: The monitor window would sometimes take a long time to load because old tasks entries were not purged from the repository.

Jira issue: RECOB-7692

Salesforce case: 119959

Type: Issue

Component/Process: Data Marts

Description: When a column was accessible by different join paths involving self-joins, ETL generation would fail with a "Duplicate columns detected" error.

Jira issue:  RECOB-7707

Salesforce case:  118694

Type: Issue

Component/Process: Data Marts, Amazon Redshift

Description: Data mart validation would sometimes fail with the following error: "Attribute used in expression for fact_shipment does not exist in the model".

Jira issue:  RECOB-7705

Salesforce case: 120120

Type: Issue

Component/Process: Data Marts, Amazon Redshift

Description: Linked dimensions to the fact table would not be displayed in the UI.

Jira issue:  RECOB-7672

Salesforce case: 114913

Type: Issue

Component/Process: BigQuery Target

Description: In certain scenarios, the list of data warehouse tables would not be not shown when clicking Display.

Jira issue:  RECOB-7728

Salesforce case:  118138

Type: Issue

Component/Process:  ETL Statements

Description: When the  the Consolidate uniform sources option was enabled, records would sometimes be missing from incremental loads.

To apply the fix, you must regenerate the data warehouse tasks.

Jira issue:  RECOB-7727

Salesforce case: 121509

Type: Issue

Component/Process: ETL Statements

Description:  ETL generation would fail when the  the Consolidate uniform sources option was enabled and the entity name was different from the table name in the landing.

Jira issue:  RECOB-7674

Salesforce case: 118869

Type: Issue

Component/Process: Data Marts, ETL Statements

Description:  Data would sometimes be missing from the data mart when dimensions were not linked to the fact table.

Jira issue: RECOB-7621

Salesforce case: 113192

Type: Issue

Component/Process: Cloudera Data Platform (CDP)

Description: A hive subquery syntax error would sometimes be encountered when generating views in GEN2.

Jira issue:  RECOB-7536

Salesforce case: 108912

Type: Issue 

Component/Process: ETL Statements

Description: The RUNNO_UPDATE column would sometimes not be updated, which would lead to missing information in the data marts.

Jira issue:  RECOB-7540

Salesforce case: 108352

Type: Issue 

Component/Process:  Data Marts, ETL Statements

Description: After a data warehouse update, records would be deleted in the data mart dimension table instead of being marked as obsolete.

Jira issue:  RECOB-7567

Salesforce case: 107411

Type: Issue

Component/Process: Data Marts 

Description: The data mart fact table would sometimes not be updated after a data warehouse update.

Jira issue:  RECOB-7587

Salesforce case: 111691

Type: Issue

Component/Process: ETL Statements

Description: Compose would mistakenly insert a RUNNO_UPDATE = 0 row when encountering an early-arriving fact.

Jira issue: RECOB-7601

Salesforce case: 111708

Type: Issue

Component/Process: ETL Statements

Description: Latest data from the change tracking landing table would not be reflected in a data warehouse with a partially mapped Type1 table.

Jira issue:  RECOB-7511

Salesforce case: 105946

Type: Issue

Component/Process:  Data Marts, ETL Statements

Description: Re-running a failed data mart task would insert duplicate rows into the dimension tables.

Jira issue:  RECOB-7570

Salesforce case: 109123

Type: Issue

Component/Process: Oracle Target

Description: Data Quality rules would sometimes generate a statement with the wrong database name.

Jira issue:  RECOB-7450

Salesforce case: 101661

Type: Issue

Component/Process: Data Marts, ETL Statements, Azure Synapse Analytics

Description: Using very complex Transactional Fact definitions in a Synapse environment would sometimes cause the following error:

SQL Statement Nested to deeply

Jira issue: RECOB-7612

Salesforce case: 114071

Type: Issue

Component/Process: ETL Statements, Oracle

Description: When the alias name "S" was defined in parentheses and then then used outside the parentheses, the following error would occur:

'ORA-00904: "S"."PROVENIENZA": invalid identifier.

Jira issue: RECOB-7527

Salesforce case: 107814

Type: Issue

Component/Process: Data Marts

Description: Data mart tasks would use the data mart schema defined in the default connection settings, even when a different schema was defined in the data mart settings.

Jira issue: RECOB-7229

Salesforce case: 79491

Type: Issue

Component/Process: UI Designer

Description: The following error would sometimes occur when editing a task's ETL SQL:

REPO-I0ITOLVER, Current item (PL_Party_assigned_....) is an older version of stored item in this repository. 

Jira issue: RECOB-7401

Salesforce case: 99326

Type: Issue

Component/Process: Data Marts

Description: In rare cases, dropping a dimension would result in repository corruption.

Jira issue: RECOB-6622

Salesforce case: 55889

Type: Issue

Component/Process: Adjust (CMPS) - Data Lakes Projects

Description: Enabling the "Save History" option for a table in the model would sometimes generate an incorrect Adjust script.

Jira issue: RECOB-7223

Salesforce case: 81724 

Type: Issue

Component/Process: Error Handling - Data Lake Projects

Description: In rare scenarios, the server log would report a "WaitForActiveTasksToFinish" error while the agent log would state "reported state done, exit value 23". The actual task log would be created empty.

Jira issue: RECOB-7485

Salesforce case: 82981

Type: Issue

Component/Process: Task Commands

Description: Backdating CDC tasks would not update the RUNNO_UPDATE value properly.

Jira issue:  RECOB-7493

Salesforce case: 105049

Type: Issue

Component/Process: Data Marts, UI Monitor

Description: Inserted rows for data marts tasks would not be visible in the monitor.

Jira issue:  RECOB-7518

Salesforce case: 107411

Type: Issue

Component/Process: Data Marts

Description: In rare scenarios, changes to data warehouse satellite tables would not be propagated to the data mart.

Jira issue: RECOB-7402

Salesforce case: 99326

Type: Issue

Component/Process: Data Marts

Description: A "Could not drop constraint" error would sometimes occur when deleting a dimension.

Jira issue: RECOB-7363

Salesforce case: 92519

Type: Issue

Component/Process: Data Marts

Description: The ETL generated for state-oriented fact tables when the environment setting 'qlk__UseClassicETLForStOrFact' was set to TRUE did not handle the OIDs for DATE and TIME dimensions.

Jira issue: RECOB-7258

Salesforce case: 89872

Type: Issue

Component/Process: UI

Description: Searching the mappings would not work as expected.

Jira issue: RECOB-7312, RECOB-7341

Salesforce case: 92519

Type: Issue

Component/Process: ETL Statements, Data Marts

Description: After upgrade, state oriented fact tables date/time OIDs would not show -1 when the date/time was not available in the date dimension.

Jira issue: RECOB-7301, RECOB-7341

Salesforce case: 90926

Type: Issue

Component/Process: Data Marts

Description: Incremental loading of dimensions would not complete as expected as the ETL query ignored changes to root level HUB and SAT tables and sometimes ignored SAT tables entirely.

Jira issue: RECOB-7026

Salesforce case: 80978

Type: Issue

Component/Process: Data Marts

Description: Data mart tasks where referencing dimensions are referring to parent dimensions that were populated with the 'Full Rebuild' option would fail with the following error:

The task was aborted as the referenced dimension 'Employees_change_Parent_DataMart' is not up-to-date. Run 'Parent_DataMart' first, and when it completes, run this task again.

Jira issue: RECOB-7264

Salesforce case: 89745

Type: Issue

Component/Process: Cloudera Data Platform (CDP)

Description: An incompatibility with Impala was discovered when using the INT function. 

Jira issue: RECOB-6597

Salesforce case: 67977

Type: Issue

Component/Process: Microsoft Azure Synapse Analytics Data Warehouse

Description: Full Load of billions of records into the data warehouse would fail with the following error: "Could not allocate tempdb space while transferring data from one distribution to another"

Loading of satellite tables has been optimized so the error no longer occurs.

Jira issue: RECOB-6643

Salesforce case: 69594

Type: Issue

Component/Process: ETL Statements

Description: When using the Data Quality rule and adding a column to the source, the data warehouse would be adjusted accordingly and the new column would be added to the data warehouse table, but Compose would not generate the new column in "Compose Error Mart" in the Task Statement.
Other enhancements:

• Support for upper case Replicate Headers
• Support for non-default Replicate header column prefixes
• The following Replicate header columns are no longer required: 'change_mask', 'stream_position' and 'operation'
• Removal of duplicates from the 'Drop error mart tables' script

Jira issue: RECOB-7217

Salesforce case: 87288

Type: Issue

Component/Process: Import from ERWin

Description: The following error would sometimes occur when importing an ERwin model: "Object reference not set to an instance of an object".

Jira issue: RECOB-7195

Salesforce case: 85992

Type: Issue

Component/Process: CLI Import - Data Marts

Description: The Compose CLI import_csv command  would fail when importing Custom ETLs.

Jira issue: RECOB-7200

Salesforce case: 86479

Type: Issue

Component/Process: Task Settings

Description: Changing the task recovery settings had no effect.

Jira issue: RECOB-7105

Salesforce case: N/A

Type: Issue

Component/Process: Security - RBAC

Description: A user with Viewer role was able to perform the CloseProjectParams operation, which should require Designer role.

Jira issue: RECOB-7292

Salesforce case: 85074

Type: Issue

Component/Process: Security

Description: Due to known security vulnerability, C:\Program Files\Qlik\Compose\java\snowflake-jdbc-3.13.14 was replaced with snowflake-jdbc-3.13.33.
See https://nvd.nist.gov/vuln/detail/CVE-2023-30535 for more details.

Jira issue: RECOB-5286

Salesforce case: 69716

Type: Issue

Component/Process: Security

Description: Due to known security vulnerability, C:\Program Files\Qlik\Compose\java\external\gson-2.8.5.jar was replaced with gson-2.8.9.jar.
See https://www.whitesourcesoftware.com/vulnerability-database/WS-2021-0419 for more details.

Jira issue: RECOB-5289

Salesforce case: 69716

Type: Issue

Component/Process: Security

Description: Due to known security vulnerability, C:\Program Files\Qlik\Compose\java\external\jackson-databind-2.12.3.jar was replaced with jackson-databind-2.15.0.jar.
See https://security.snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-2421244 for more details.

Jira issue: RECOB-7193

Salesforce case: 85074

Type: Issue

Component/Process: Security

Description: JQuery JS versions with vulnerabilities remained in the Compose installation folder instead of only the latest version.

Jira issue: RECOB-6985

Salesforce case: 76987

Type: Issue

Component/Process: CLI

Description: The command to reset the watermarks table would not work.

Jira issue: RECOB-7124

Salesforce case: N/A

Type: Issue

Component/Process: Data marts

Description: For dimensions with satellites containing only foreign keys, changes to the foreign keys would not be captured.

Jira issue: RECOB-7123

Salesforce case: 83371

Type: Issue

Component/Process: Databricks on Azure

Description: It would not be possible to create column names with special characters and spaces.

Jira issue: RECOB-7072

Salesforce case: 82981

Type: Issue

Component/Process: Data Marts, ETL Statements, Performance

Description: Data Mart queries for building Type 2 dimensions would take a long time to complete in Snowflake. After installing this patch, you need to re-generate the task ETLs.

To revert to the previous behavior, set the environmental variable "qlk__PreselectT2DimIDs" to 'false' or '0'.

Jira issue: RECOB-6774

Salesforce case: 73478

Type: Issue

Component/Process: Data Marts, ETL Statements, Performance

Description: During incremental loading, the ETL for loading Type 2 dimensions would fetch all rows from the hub and satellite tables instead of only the updated rows. After installing this patch, you need to re-generate the task ETLs.

To revert to the previous behavior, set the environmental variable "qlk__PreselectT2DimIDs" to 'false' or '0'.

Jira issue: RECOB-6994

Salesforce case: 80785

Type: Issue

Component/Process: Data Marts - Snowflake

Description: When different databases were defined for the landing and the data warehouse, the 'Evaluate Expression' would be performed on the wrong database.

Jira issue: RECOB-7025

Salesforce case: 81154

Type: Issue

Component/Process: ETL Statements

Description: Evaluated values would be truncated to type of UNSIGNED INTEGER, resulting in negative results in the event of an overflow. This has been fixed to use type LONG instead.

Jira issue: RECOB-7103

Salesforce case: 84139

Type: Issue

Component/Process: Data mart adjust

Description: Adding a referenced Type 2 dimension to a star schema would result in incorrect adjustment. The fix drops and recreates the data mart instead of adjusting it.

Jira issue: RECOB-7108

Salesforce case: 84140

Type: Issue

Component/Process: CLI

Description: The "ComposeCli.exe generate_tasks" command would not generate a specific data mart task.

Jira issue: RECOB-7086

Salesforce case: 83519

Type: Issue

Component/Process: Data Marts

Description: The following error would occur during ETL generation of data marts with reference-dimensions:

Object reference not set

Jira issue: RECOB-6845

Salesforce case: 76209

Type: Issue

Component/Process: Data Marts

Description: Generating the data mart task would sometimes fail with the following error:

SYS,GENERAL_EXCEPTION,Sequence contains no elements

When validating a data mart, the following error would sometimes occur:

Failed to Find dimension. Fact/dimension <fact name> object include wrong Path property

This error occured when the fact definition contained an OID-column pointing to non-existing dimensions. The issue was resolved by automatically dropping the OID-columns from the fact table in such situations. However, users need to review the "non-existing" dimensions and remove if necessary.

Jira issue: RECOB-7024

Salesforce case: 81584

Type: Issue

Component/Process: Data Marts

Description: An error sometimes occurred when generating the data mart task.

Jira issue: RECOB-6755

Salesforce case: 73291

Type: Issue

Component/Process: Data Marts

Description:  Deleting a referenced dimension would sometimes result in an error.

Jira issue: RECOB-7016

Salesforce case: 80287

Type: Issue

Component/Process: Data warehouse ask generation

Description: Generating the data warehouse CDC task would sometimes fail with the following error:

SYS,GENERAL_EXCEPTION,Sequence contains no elements

Jira issue: RECOB-7048

Salesforce case: N/A

Type: Enhancement

Component/Process: Performance

Description: Added an option to improve performance of History and Live History views, but without filtering out potential duplicates. This improvement is controlled using an environment variable.

Jira issue: RECOB-7002

Salesforce case: 80613

Type: issue

Component/Process: Permissions

Description: Compose would not allow users with the Designer role to associate a Replicate task or choose a task from a Replicate server.

Jira issue: RECOB-7021

Salesforce case: 81149

Type: Issue

Component/Process: ETL statements, Data marts

Description:  The generated data mart task statements would sometimes contain runtime clauses with unbalanced brackets.

Jira issue: RECOB-6962

Salesforce case: 76585

Type: Issue

Component/Process: Update Statistics on Microsoft Azure Synapse Analytics

Description: The following 'Update Statistics' issues have been resolved:
1) When the system variable 'qlk__UpdateStatisticsPercentageDma' was set to 0, a runtime error would occur.
2) For Hub and Transactional fact tables, statistics were not updated.

Jira issue: RECOB-6973

Salesforce case: 80395

Type: Issue

Component/Process: Data mart ETL statements on Microsoft Azure Synapse Analytics and Microsoft SQL Server

Description: Loading State-Oriented fact tables would take an excessively long time when numerous dimensions were involved. The issue was resolved by using an temporary table that stores intermediate results. In certain environments the fix might not have the desired effect. In this case, you can revert to the previous method of loading State-Oriented fact tables by setting the environmental variable 'qlk__UseClassicETLForStOrFact' to '1' or 'true'. After setting the environment variable, you need to restart the Qlik Compose service.

Jira issue: RECOB-6968

Salesforce case: 80259

Type: Issue

Component/Process: Cli 

Description: When long CLI commands were run by non-admin users, the following error would sometimes occur:

Failed to send a ping message, SYS-E-HTTPFAIL, SYS-E-AUTHERR, Unauthorized Request. request: Ping.

Jira issue: RECOB-6876

Salesforce case: 70246

Type: Issue

Component/Process: Security

Description: Parsing of expressions and input attribute references would not work for non-admin users that had admin permissions on a specific project.

Jira issue: RECOB-6903

Salesforce case: N/A

Type: Issue

Component/Process: Model validations

Description: It would not be possible to use Change Table views in a mapping when Keep changes was turned on in the landing task settings.

Jira issue: RECOB-6916

Salesforce case: N/A

Type: issue

Component/Process: Project deployment

Description: When deploying a project for the first time (which means the data warehouse connection was configured without password), and the Create DDL scripts only option was enabled in the project settings, an "object reference not set" error would occur when creating tables.

Jira issue: RECOB-6917

Salesforce case: N/A

Type: issue

Component/Process: Discovery and mapping

Description: Discovering from a source would create two duplicate mappings per entity, instead of only one mapping as it should have.

Jira issue: RECOB-6858

Salesforce case: 76867

Type: Issue

Component/Process: Data mart validation

Description: Resolves the following issues:

• When a fact or dimension contained two attributes with the same name, but a different prefix, importing a data mart from a CSV file would fail during validation with a "Multiple attributes (n = 2) use the same internal ID" error.
• n error would sometime occur when a fact and a dimension had the same name.
• When deleting a data mart with a custom ETL, the internal object would not be deleted. This would cause issues when performing additional imports.
• When performing an import, if a custom ETL with same name exists in another data mart, the imported data mart will be appended with a suffix and imported instead of being skipped.

•  

Jira issue: RECOB-6828

Salesforce case: 76586

Type: Issue

Component/Process: Compose CLI

Description: The CLI would generate an incorrect statement for adding statistics to aggregated fact tables. 

Jira issue: RECOB-6734

Salesforce case: 72085

Type: Issue

Component/Process: Data Mart Validation

Description: In rare scenarios, validating a data mart would mistakenly cause columns with names ending in 'OID' to be dropped.

Jira issue: RECOB-6746

Salesforce case: N/A

Type: Issue

Component/Process: Azure Synapse Analytics - Project Deployment

Description: Deploying an Azure Synapse Analytics project would take an excessive amount of time.

Jira issue: RECOB-6570

Salesforce case: 66314

Type: Issue

Component/Process: Consolidation 

Description: When using the Consolidate uniform sources option with archiving enabled, Compose would generate an invalid SQL statement during the CDC ETL process.

Jira issue: RECOB-6687

Salesforce case: 67429

Type: Issue

Component/Process: Schema Evolution - Databricks

Description: Schema Evolution would adds a STRING column as a VARCHAR.

Jira issue: RECOB-6570

Salesforce case: 66314

Type: Issue

Component/Process: ETL Generation with Consolidation

Description: When the Consolidation uniform sources option is enabled, a check will now be done that prevents generating faulty ETL code when source tables/queries are missing. The check provides information so that users can resolve the issue  on their own.

Jira issue: RECOB-6776

Salesforce case: 74070

Type: Issue

Component/Process: Upgrade - Snowflake

Description:  After upgrading to Compose 2022.5, an error would occur on Snowflake when the connection string contained additional parameters.

Jira issue: RECOB-7071 

Salesforce case: N/A

Type: Issue

Component/Process: Incremental load

Description:  When performing incremental load of Type 2 data, records would sometimes be deleted on Microsoft Azure Synapse Analytics and Snowflake platforms.

Jira issue: RECOB-6639

Salesforce case: 69870

Type: Issue

Component/Process: Data Mart Generation

Description: The following error would sometimes be encountered when trying to generate a data mart:

GENERAL_EXCEPTION,Object reference not set to an instance of an object​

Jira issue: RECOB-6630

Salesforce case: 69276

Type: Issue

Component/Process: Data Mart Tasks

Description: An "invalid column name" error would sometimes be encountered when running data mart tasks.

Jira issue: RECOB-6619

Salesforce case: 67621

Type: Issue

Component/Process: Data Mart Tasks

Description: When regenerating tasks, Compose would automatically adjust all facts tables by removing the Date dimension OID​.

Jira issue: RECOB-6514

Salesforce case: 69682

Type: Issue

Component/Process: Data Marts

Description:  The following error would sometimes occur in an Aggregated Star Schema with Date Dimension​:

Column  is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause

Jira issue: RECOB-6537

Salesforce case: 66261

Type: Issue

Component/Process: Data Marts Tasks

Description: The following error would sometimes occur when generating data mart tasks.

Sequence contains no elements​

Jira issue: RECOB-6453 + RECOB-6481

Salesforce case: 63783 + 63736

Type: Issue

Component/Process: Data Marts Tasks

Description: The following error would sometimes occur when generating data mart tasks:

Value Cannot be Null

Jira issue: RECOB-6635

Salesforce case: 69134

Type: Issue

Component/Process: Data Marts Tasks

Description: An "Invalid identifier" error would sometimes occur when running data mart tasks:

Jira issue: RECOB-5606

Salesforce case: 42279

Type: Issue

Component/Process: Custom ETL

Description: When opening a Custom ETL, code with more than 11 lines would not load completely.

Jira issue: RECOB-6511

Salesforce case: 36345

Type: Issue

Component/Process: Compose Import Project Repository CLI

Description: When running the import_project_repository​ command, the following error would sometimes be encountered:

Data mart is not valid

Jira issue: RECOB-6527

Salesforce case: 65766

Type: Issue

Component/Process: Logging

Description: When a data mart task instance failed, the data mart log information would sometimes be inaccurate.

Jira issue: RECOB-6611

Salesforce case: 67579

Type: Issue

Component/Process: Compose Export Project CLI

Description: The --exclude_envar_values parameter would not work as expected.

Jira issue: RECOB-6674

Salesforce case: 70086

Type: Issue

Component/Process: Data Warehouse Generation

Description: Data warehouse ETL generation would fail when query-based mapping included custom environment variables.

Jira issue: RECOB-6691

Salesforce case: 71770

Type: Issue

Component/Process: Data Mart

Description: Expressions would be ignored when defined on existing fact table attributes.

Jira issue: RECOB-6433

Salesforce case: N/A

Type: Enhancement

Component/Process: Apache Impala views in Data Lakes projects

Description: The header__batch_modified column will now be cast as varchar(32) for the outbound Apache Impala views. To leverage this enhancement, you need to set an environment variable.

Jira issue: RECOB-6508

Salesforce case: 63304

Type: Issue

Component/Process: AWS glue in Data Lakes projects

Description: ​​CDC tasks using AWS glue would sometimes fail with the following error:

expects to have a column name on a left side, but got 'substr"​

Jira issue: RECOB-6568

Salesforce case: 54618

Type: Issue

Component/Process: Hive 3.1.3 in Data Lakes projects

Description: ​​When using Hive 3.1.3, the following error would sometimes be encountered:

SemanticException Line 0:-1 Wrong arguments 'hdr__ts': Casting DATE/TIMESTAMP types to NUMERIC is prohibited

Jira issue: RECOB-6566

Salesforce case: N/A

Type: Issue

Component/Process: Upgrade from Compose for Data Lakes 6.6

Description: Upgrading from Compose for Data Lakes 6.6 would cause the column prefix to change from header to hdr. To leverage this fix, you need to set an environment variable.

Jira issue: RECOB-5343

Salesforce case: 35725

Type: Issue

Component/Process: Discovery

Description: It would not be possible to discover landing views.

Jira issue: RECOB-6072

Salesforce case: 54081

Type: Issue

Component/Process: Compose CLI

Description: The compare_csv CLI option would not work properly when project items contained line breaks. 

Jira issue: RECOB-6268

Salesforce case: 59331

Type: Issue

Component/Process: Data Mart Generation

Description: ​​The following error would sometimes occur when generating a data mart:

Multiple attributes (n = 2) use the same internal ID: 'DESCRIPTION', 'TEST_NAME'

Jira issue: RECOB-6282

Salesforce case: 58371

Type: Issue

Component/Process: Tasks

Description: ​​When a project had two full load tasks reading from the same landing zone database, the CDC task would start from the default partition.

Jira issue: RECOB-5808

Salesforce case: 48291

Type: Issue

Component/Process: Data Mart (Fact) Performance

Description: ​​Performance issues would be encountered when updating the fact table.

Jira issue: RECOB-6356

Salesforce case: 60650

Type: Issue

Component/Process: Data Mart Tasks

Description: Data mart loading would sometimes fail at the "Merging changes into dimension" stage.

Jira issue: RECOB-6410

Salesforce case: 62525

Type: Issue

Component/Process: Environment Variables - Data Warehouse Projects

Description: Applying a predefined variable using the CLI would result in an "Object reference not set to an instance of an object" error.

Jira issue: RECOB-6435

Salesforce case: N/A

Type: Issue

Component/Process: Data Mart Tasks

Description: When working with Microsoft Azure Synapse, it would not be possible to create a data mart in a non-default schema.

Jira issue: RECOB-5290

Salesforce case: N/A

Type: Issue

Component/Process: Security

Description: Updated the ojdbc component (ojdbc7-12.1.0.2) to the newest version, which fixes security vulnerability CVE-2016-3506.

Jira issue: ​RECOB-5285

Salesforce case: N/A

Type: Issue

Component/Process: Security

Description: Updated the PostgreSQL component (postgresql-42.2.25) to the newest version, which fixes security vulnerability CVE-2022-21724.

Jira issue: RECOB-6409

Salesforce case: N/A

Type: Issue

Component/Process: Microsoft Azure Synapse Analytics

Description: If the connection to the database was aborted, the task would not recover.

Jira issue: RECOB-6067

Salesforce case: N/A

Type: Issue

Component/Process: Microsoft Azure Synapse Analytics

Description: Changing dimensions from Type 2 to Type 1 would sometimes result in the following errors when recreating the associated tables:

SYS,GENERAL_EXCEPTION,Column name '<column-name>' does not exist in the target table or view

Jira issue: RECOB-6267

Salesforce case: N/A

Type: Issue

Component/Process: Data Mart Validation

Description: When validating a data mart with Date and Time columns, the validation would incorrectly report the following message:

The data mart tables in the database are different from the data mart definition

Jira issue: RECOB-6439

Salesforce case: N/A

Type: Issue

Component/Process: Project Import

Description: After importing a project from a diagnostic package, editing the connection settings would result in a 'SYS,DESERIALIZE_TO_TYPE' incompatibility error.

Jira issue: RECOB-6307

Salesforce case: 59873

Type: Issue

Component/Process: Data Mart Task

Description: Due to an issue with handling relationships, the data mart task would sometimes fail with the following error:

Invalid column name <name>

Downloads

Download this release from the Product Downloads page on Qlik Community.

About Qlik

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.

qlik.com