Skip to main content

Qlik

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!

Qlik Catalog Release Notes - May 2022 Initial Release to Service Release 1

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
djenkins-qlik
Employee
Employee

Qlik Catalog Release Notes - May 2022 Initial Release to Service Release 1

Last Update:

Oct 5, 2022 3:48:39 AM

Updated By:

frida_tell

Created date:

Oct 4, 2022 3:59:58 PM

Table of Contents

 

The following release notes cover the versions of Qlik Catalog released in May 2022. 

What's new in Qlik Catalog May 2022

  • Publish to Qlik Sense: Multiple Sense server support in user interface
  • Lineage API additions to enable updating "other" nodes and edges (e.g., rename)
  • New Sources auto-configure to allow non-ascii and control characters
  • New installs automatically enable Tomcat HTTPS and NextGen XML
  • Post-install script to auto-configure container services
 
Publish to Qlik Sense: Multiple Sense Server Support in User Interface

Multiple Sense servers may now be configured for Publish to Qlik Sense. Specific Sense servers are configured using the Catalog user interface, along with traditional QVD Import settings (see menu Admin / Qlik Sense). Only global Publish to Qlik Sense settings remain in file core_env.properties. When Publish to Qlik Sense is selected for QVD entities, the Sense server from which the QVDs originated is automatically selected. When non-QVD entities are selected, a new dialog is shown and the user must pick the Sense server on which the app will be created/updated.

Please see the online help, as well as the "Qlik Catalog + Qlik Sense Integration Guide", for more detail. In addition, impacted core_env properties are listed in the appendix.

Dialog to select Qlik Sense server when publishing non-QVD entities:

Picture1.png

Dialog to configure Publish to Qlik Sense (along with QVD Import):

Picture2.png

Lineage API Additions to Enable Updating "other" Nodes and Edges

Two additional update (PUT) calls were added to the lineage API:

  • update node label (aka name) and metadata
  • update edge metadata

Only nodes and edges of type "other" may be updated. These are typically used to illustrate upstream lineage ancestry (and are created/deleted using the API). Until this enhancement, nodes could not be renamed.

Please visit the "live" documentation included with Catalog at Support / API Documentation.

 

New Sources Auto-Configure to Allow Non-Ascii and Control Characters

Going forward, newly created Sources (of all types except MAINFRAME) will automatically be configured to allow non-ascii and control characters. A common use case that will benefit is loading JDBC (relational) data containing foreign characters. These loads will now become "zero-config".

Because these properties are set at the Source level, they may be overridden at the Entity level. To revert to legacy behavior, either delete the properties or set them to false (see screenshot). Note that the Discover module version of the Source also has these properties set. Prepare Dataflow target Entities added to these Sources will also pick up the settings.

Picture3.png

New Installs Automatically Enable Tomcat HTTPS and NextGen XML

Going forward, new Catalog installs are automatically configured to use HTTPS (port 8443). HTTP (port 8080) is automatically redirected to HTTPS. The NextGen XML feature (used to load XML data) is automatically configured to work alongside (a dedicated HTTP port, 8082, is configured for it).

 

Post-Install Script to Auto-Configure Container Services

A new script, QDCpostinstall.sh, may be run after installing Catalog -- it configures all containers as auto-start Linux services.

There are now three scripts that can be used for the entire installation process:

  • QDCprereqs.sh -- run as root, installs all prerequisite software, creates qdc account, etc.
  • QDCinstaller.sh -- run as service user (qdc), installs Catalog
  • QDCpostinstall.sh -- run as root, configures Linux services

Please see the installation guide for more details.

 

Resolved Defects

  • QDCB-1148 - Fixed Incorrect Prepare Dataflow Script Generation
  • QDCB-1149 and QDCB-1150 - Improved Qlik Sense Integration Scripts
  • QDCB-404 - Fixed "Host" Header Poisoning
  • QDCB-1154 - Fixed Install to Alternate Home Directory
  • QDCB-1143 - Fixed Double-escaping of Backslash When Loading Data

  • QDCB-1139 - Addressed Spring Framework Vulnerabilities

 

May 2022 SR1 (4.14.1)

Improved Qlik Sense Integration Scripts

Jira ID: QDCB-1149 and QDCB-1150

The Node.js scripts used to integrate Qlik Catalog with Qlik Sense where enhanced to uniformly:

  • perform request authorization -- not all scripts validated that the Qlik Sense SSL certificate was signed by the root.pem CA certificate copied to the Catalog server
  • configure a network timeout -- many scripts did not implement a timeout handler, leading to timeouts of up to two minutes
  • handle hostname mismatch -- for some scripts, return value ERR_TLS_CERT_ALTNAME_INVALID was seen if the host which was connected to was not in the SSL certificate's altnames

To disable enhanced authorization, edit /usr/local/qdc/qlikpublish/securityUtils.js and change the following to false:

exports.rejectUnauthorized = true;
exports.rejectUnauthorizedForCreateAndUpdateApp = true;

 

Fixed Incorrect Prepare Dataflow Script Generation

Jira ID: QDCB-1148

Prepare Dataflow execution failed due to incorrect script generation for certain entity fields used in Aggregation functions. If an entity contained two fields (e.g., QTY_BASE_PRICE_SIGN and BASE_PRICE), where one field was a substring of the other, script generation was incorrect if the longer of the two fields was used in an Aggregation function (a double replacement occurred).

Fixed "Host" Header Poisoning

Jira ID: QDCB-404

An attacker could craft a request to Catalog containing an HTTP Host header value of a site they control causing Catalog to return a page with references to their malicious site. HTTP caching mechanisms between Catalog and its clients can become poisoned with malicious content serving the attacker's page in place of Catalog's intended content.

The Host header value should not automatically be trusted by Catalog. Instead, a protected configuration setting should be used to specify the desired value. A new core_env property is introduced:

# To protect against HTTP headers such as "Host" and "X-Forwarded-Host" being abused by a malicious client, set this

# to the hostname used by clients. It is embedded in link results returned by Catalog searches (e.g., /catalog/search).

# Restart required. Default: not set.

external.hostname=<FULLY_QUALIFIED_DOMAIN_NAME>

For new installs, this property is initialized to the result of Linux command "hostname". Upgrades must manually add this property. Also, upgrades must manually update webapps/qdc/WEB-INF/web.xml and add the following before the closing "</web-app>" tag:

<filter>

        <filter-name>ServerNameRequestFilter</filter-name>

        <filter-class>com.nvs.ui.security.ServerNameRequestFilter</filter-class>

    </filter>

    <filter-mapping>

        <filter-name>ServerNameRequestFilter</filter-name>

        <url-pattern>/*</url-pattern>

        <dispatcher>REQUEST</dispatcher>

        <dispatcher>FORWARD</dispatcher>

    </filter-mapping>

To specify legacy behavior, either remove the filter definition from webapps/qdc/WEB-INF/web.xml or do NOT set the core_env property 'external.hostname'.

Fixed Install to Alternate Home Directory

Jira ID: QDCB-1154

Catalog was not correctly configured when an alternate home directory (e.g., /apps/qdc rather than /usr/local/qdc) was specified. This has been fixed. To install to an alternate home directory, change QDC_HOME and run the following:

  • QDCprereqs.sh -- change QDC_HOME and run as root/sudo user
  • QDCinstaller.properties -- change QDC_HOME and then run QDCinstaller.sh as service account (qdc)
  • QDCpostinstall.sh -- change QDC_HOME and run as root/sudo user

Please see the install guide for more information about editing and executing these files.

May 2022 Initial Release (4.14)

Fixed Double-escaping of Backslash When Loading Data

Jira ID: QDCB-1143

If customer data contained escaped characters (e.g., \t), when loading data (under specific configurations) Catalog would add a second backslash (e.g., \\t). If then run through a Prepare Dataflow, the backslashes were again escaped (e.g., \\\\t). This issue has been fixed.

 
Addressed Spring Framework Vulnerabilities

Jira ID: QDCB-1139

This release addresses the following Spring Framework vulnerabilities:

  • CVE-2022-22965 - Spring Framework RCE via Data Binding on JDK 9+
  • CVE-2022-22950 - Spring Expression DoS Vulnerability

No Longer Supported

  • With the August 2021 release, CDH and HDP Hadoop cluster environments are no longer supported (supported Hadoop cluster environments are AWS EMR and CDP Private Cloud 7)
  • With the May 2021 release, Microsoft Internet Explorer 11 is no longer supported and Workflow Scheduler was removed

 

Upgrade notes

Migrating to or Upgrading Tomcat 9

Beginning with the May 2021 release, only Apache Tomcat 9 is supported. The installer will prohibit other versions. If using Tomcat 7, please first initiate a migration to Tomcat 9 before installing this release. Then, when installing, the upgrade option (-u) is NOT used.

These instructions may also be used to upgrade from an older version of Tomcat 9 to a newer version.

Step

Sample Commands

Shutdown and rename old Tomcat 7 or 9

cd /usr/local/qdc (or cd /usr/local/podium)

./apache-tomcat-<OLD_VERSION>/bin/shutdown.sh

mv apache-tomcat-<OLD_VERSION> old-apache-tomcat

Download and expand Tomcat 9 - NOTE: adjust version 9.0.62 to use latest 9.0.x series

wget https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.62/bin/apache-tomcat-9.0.62.tar.gz

tar -xf apache-tomcat-9.0.62.tar.gz

rm apache-tomcat-9.0.62.tar.gz

Copy core_env.properties from old Tomcat to new Tomcat 9

cp old-apache-tomcat/conf/core_env.properties apache-tomcat-9.0.62/conf/

If migrating from Tomcat 7: Extract server.xml from podium.zip and copy to new Tomcat

unzip -j podium-4.<VERSION>-<BUILD>.zip podium/config/tomcat9-server.xml -d .

mv ./tomcat9-server.xml apache-tomcat-9.0.62/conf/server.xml

If upgrading Tomcat 9: Copy server.xml from old Tomcat 9 to new Tomcat 9

cp old-apache-tomcat/conf/server.xml apache-tomcat-9.0.62/conf/

If the old Tomcat 9 was configured for HTTPS, and the keystore (jks file) was stored in the old Tomcat directory, migrate it to the new Tomcat directory, and update conf/server.xml to reference it. Consider placing the keystore file in a non-Tomcat directory such as /usr/local/qdc/keystore.

Configure QDCinstaller.properties for Tomcat 9

Whether using an existing QDCinstaller.properties file from a previous install, or configuring one for the first time, ensure that it is updated to point to Tomcat 9:

TOMCAT_HOME=/usr/local/podium/apache-tomcat-9.0.62

Finally, when the installer is run, do NOT specify upgrade mode (-u), as some files should be created as if it were a first-time install.

./QDCinstaller.sh

At this point, Tomcat 9, if newly installed, will support only HTTP on port 8080.

Verify successful Qlik Catalog startup and basic functionality.

Additional configuration will be required to enable HTTPS on port 8443, apply security headers, etc. If Tomcat 7 used HTTPS, the keystore (jks file) containing the public-private keypair should be copied to Tomcat 9 and conf/server.xml updated.

In addition, Tomcat 7 may have been configured as a service. It should be disabled. Tomcat 9 may be configured as a service to automatically start.

Please see the install guide for guidance on both.

 

 

Process if Upgrading From June 2020 or Earlier

Do not attempt to upgrade until the following is understood


If upgrading from a version of Qlik Catalog prior to September 2020 (4.7) there are utilities that MUST be run after Catalog is upgraded. Once run, the utilities need never be run again.

The server may not start until the first two utilities have been run and will log a WARN at startup until the third is run. Do NOT upgrade the server until familiar with these utilities and the information required to run them. It will take time to gather this information. Gathering the information BEFORE Catalog is upgraded will minimize downtime.

Run the utilities in this order:

  1. jwt2CertsUtility -- please review readme.txt
    This will be required if Qlik Sense Connectors have been defined to load QVDs.
    Will need to gather networking info and certificate files from Qlik Sense servers.
    May be run from any directory.

 

  1. singleNodeUpgradeForEntitiesWithBadOrUglyData.sh -- please review comment in script
    This will be required only if the installation is single node.
    Will need podium_dist database info if defaults altered.
    May be run from any directory.

 

  1. singleNodeUpgradeToGrantReadOnlyUserAccessToDistSchemas.sh -- please review comment in script
    This will be required only if the installation is single node.
    Will need podium_dist database info if defaults altered.
    May be run from any directory.

 

Log of Changes to File core_env.properties

A chronological listing (most recent first) of additions, changes in behavior, and deletions to the primary global configuration file, core_env.properties.

 
May 2022 SR1

ADDITION: Fixed "Host" Header Poisoning

The Host header value should not automatically be trusted by Catalog. Instead, a protected configuration setting is introduced. See earlier description of QDCB-404.

# To protect against HTTP headers such as "Host" and "X-Forwarded-Host" being abused by a malicious client, set this

# to the hostname used by clients. It is embedded in link results returned by Catalog searches (e.g., /catalog/search).

# Restart required. Default: not set.

external.hostname=<FULLY_QUALIFIED_DOMAIN_NAME>

 
May 2022

CHANGE: Enable Publish to Qlik Sense by Default

Publish to Qlik Sense will now be enabled by default.

# Setting this to true will start showing 'Publish to Qlik' option in Podium UI cart checkouts.

# Default: true

is.publish.to.qlik.enabled=true

DELETION: Publish to Qlik Sense: Multiple Sense Server Support in User Interface

Configuration of specific Sense servers for Publish to Qlik Sense has been moved to the Catalog user interface. General settings remain in core_env.properties.

The following is taken from the "Qlik Catalog + Qlik Sense Integration Guide". The nine, new UI fields for Publish to Qlik Sense were formerly in core_env.properties. These properties can now be retired (either commented-off or removed from core_env.properties):

  • qlik.sense.url
  • qlik.sense.redirect.url
  • qlik.sense.enginePort
  • qlik.sense.proxyPort
  • qlik.sense.active.directory.name
  • podium.qlik.username
  • qlik.sense.skip.ticket
  • qlik.sense.invert.security.model
  • podium.qlik.dataconnection.name

The following two properties were also removed and have been superseded by pre-existing Qlik Sense Connector fields:

  • qlik.sense.root.admin.directory.name (superseded by Admin User Directory)
  • qlik.sense.root.admin.user.name (superseded by Admin User Name)

The following global settings remain in the core_env.properties file -- please see the comments there:

  • is.publish.to.qlik.enabled
  • podium.qlik.dataconnection.managed.entity.generating.from
  • podium.qlik.dataconnection.addressed.registered.entity.generating.from
  • podium.qlik.replace.previous.script
  • qlik.sense.data.deletion.deferral.period.hours
  • podium.qlik.dataconnection.sql.openquote.char
  • podium.qlik.dataconnection.sql.closequote.char
  • podium.qlik.appcreation.nodecommand
  • podium.qlik.qsockclient.script.directory
  • qlik.logs.purging.threshold
  • qlik.sense.record.count.threshold.for.data.load
 
February 2022 SR1

CHANGE: Enable Catalog Auto-Creation of Sense Security Rules & Auto Add Newly Discovered QVDs to Local/AD Groups

The property controlling these capabilities is now named "qlik.sense.invert.security.model". It was formerly named "qlik.sense.auto.create.security.rules" when introduced in the initial February 2022 release. See earlier description of QDCB-1114.

# Normally, on QVD import, Catalog creates Groups that shadow Sense Data Connections. QVD entities are then added to
# these Groups. A Sense admin creates Security Rules granting Users access to Data Connections. When a User logs

# in to Catalog, a security audit is conducted against Sense, and the User is added to Groups if they have access to the

# corresponding Data Connections.

#

# This model may be inverted. A Catalog admin can instead manually add QVD entities to Catalog local/AD groups. When

# this occurs, users running Publish to Qlik Sense may need Data Connection Security Rules created in order to load

# published data in Sense. Catalog can be configured to automatically create any needed Security Rules by setting this

# property to true. In addition, this property ensures that once a single QVD entity has been manually added to a Group,

# future QVD entities discovered during import in the same Data Connection folder will be automatically added to the

# same group.

#

# Properties 'qlik.sense.root.admin.directory.name' and 'qlik.sense.root.admin.user.name' must also be set.

# Formerly: qlik.sense.auto.create.security.rules. Default: false

#qlik.sense.invert.security.model=true

 
February 2022

ADDITION: Enable Catalog Auto-Creation of Sense Security Rules

Set this property to true to have Catalog auto-create Qlik Sense Data Connection Security Rules (if needed) as part of the Publish to Qlik Sense process. See earlier description.

# If Catalog local or AD Groups have had QVD Sources/Entities added to them, users running Publish to Qlik Sense

# may need Data Connection Security Rules created in order to load data in Sense. Normally, these Security Rules

# should be created and managed in Sense QMC. However, Catalog can be configured to automatically create any

# needed Security Rules if this property is set to true. Properties 'qlik.sense.root.admin.directory.name' and

# 'qlik.sense.root.admin.user.name' must also be set. Default: false

#qlik.sense.auto.create.security.rules=true

ADDITION: Alter SAML Identity Provider User Domain Name

Specify this property to set or alter the domain name of the user sent by the SAML identity provider (IdP) to Catalog. See earlier description.

# If the IdP does not append a domain and one is needed, or you wish to change the domain, set this property. If the

# property is set to a value of "test.com", a SAML principal name of "jdoe" would become "jdoe@test.com"; a SAML

# principal name of "jdoe@other.com" would become "jdoe@test.com". The known use case for this property is to match

# principal (aka user) names coming from the IdP with the users imported from Active Directory. Default: not set

#saml.alternate.domain=

 
November 2021 SR2

No changes.

 
November 2021 SR1

CHANGE: Publish to Qlik Sense Enhancement

The Publish to Qlik Sense RootAdmin user is now more widely applied. Before, it was only used when multiple domains were specified in property "qlik.sense.active.directory.name". Now, it is used to ensure a known, valid Sense user is being used for Publish to Qlik Sense. See earlier description of QDCB-1007.

# Enter the directory and user name of a Sense 'RootAdmin' user.

# Used to validate that the domain user being used for Publish to Qlik Sense has previously logged into the

# Sense server. This prevents users known only to Catalog being inadvertently created in Sense.

# Mandatory if multiple directories were specified in property 'qlik.sense.active.directory.name'.

qlik.sense.root.admin.directory.name=AD

qlik.sense.root.admin.user.name=sense-service

ADDITION: Extended Support for Fields in Prepare Dataflows That Are Also Pig Reserved Words

Prepare dataflow jobs will fail if fields are named using unanticipated Apache Pig reserved words. Such words can now be configured. See earlier description of QDCB-1107.

# Entity fields used in Prepare Dataflows may also be Pig reserved words (e.g., STORE). Frequently used reserved words

# are correctly handled if they are field names. This property may be used to augment the set of known reserved words

# with unanticipated words. Words must be comma separated. Default: not used

#pig.reserved.words.additional=register,CASE

 

Downloads

May 2022 SR1 Catalog Application

May 2022 SR1 Catalog Installer

 

 

About Qlik

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.

qlik.com

Labels (1)
Labels:
389 Views
Release details
Products:
Qlik Catalog
Release name:
May 2022
Patch:
1
Language:
English
Version history
Last update:
‎2022-10-05 03:48 AM
Updated by:
Former Employee