Table of Contents
The following release notes cover the versions of Qlik Catalog released in August 2021.
What's new in Qlik Catalog August 2021 Service Release 2
Noteworthy
- Ubuntu 20.04 and Red Hat Enterprise Linux (RHEL) 8 Support
- AWS EMR 5.33 and CDP Private Cloud 7 Support
- SSL-Enabled PostgreSQL
No longer supported
- With the initial August 2021 release, CDH and HDP cluster environments are no longer supported
Ubuntu 20.04 and Red Hat Enterprise Linux (RHEL) 8 Support
Catalog may now be installed on Ubuntu 20.04 and RHEL 8. On these platforms, podman (a tool for managing images and containers) is required. Please see either the single- or multi-node installation guide.
Catalog continues to support CentOS 7 and RHEL 7. On these platforms, docker and docker compose are required. Please see either the single- or multi-node installation guide.
AWS EMR 5.33 and CDP Private Cloud 7 Support
The multi-node version of Catalog may now be deployed on proper edge nodes of EMR 5.33 and CDP 7 cluster environments. Please see the multi-node installation guide.
SSL-Enabled PostgreSQL
The custom build of PostgreSQL (enabling support for long schema/source and table/entity names) distributed with Catalog is now compiled with the components required to enable SSL for incoming connections. Once SSL is enabled for PostgreSQL, it will listen for both normal and SSL connections on the same TCP port and will negotiate with any connecting client (including Catalog) on whether to use SSL. Follow PostgreSQL 11 documentation for configuring and enabling SSL (https://www.postgresql.org/docs/11/ssl-tcp.html).
Do NOT attempt enablement if you are not comfortable with the database administration of PostgreSQL.
Please request draft documentation of using Catalog with FIPS and PostgreSQL SSL.
Resolved Defects
August 2021 SR2 (4.11.2)
Address log4j2 Zero-Day Exploits, CVE-2021-44228, CVE-2021-45046 and CVE-2021-45105
Jira ID: QDC-1285
Single-node Catalog has been upgraded to use log4j2 version 2.17.0.
August 2021 SR1 (4.11.1)
Global Search Filtering Fixed
Jira ID: QDCB-1024
Global search, available at the top-right of all modules except Catalog, now correctly supports filtering which fields are searched. Any one of the following four fields may be checked/unchecked to indicate if the field should be searched:
Filters: x Name x Tags x Business Name x Business Description
The selection of no fields is the same as the selection of all fields. The "Search All" option has been removed -- simply check or uncheck all of the fields. It's logical to only select one, two or three fields.
The checked filter fields are transferred to the "View all results" page when that link is clicked.
Support for Global Search Using % Wildcard
Jira ID: QDCB-1051
Every search (except the Catalog module search) would return an error if the "%" wildcard character was included in the search term. For example, global search, entity search, audit log search, etc., were all broken if text like "%day" was input. "%" is the SQL multi-character wildcard. As an aside, the SQL single-character wildcard "_" is also supported.
The issue arose because the search term was embedded in the URL path. An encoded "%" (i.e., "%25) can no longer be embedded in a URL path element. This change took effect in the May 2021 release when Catalog upgraded a library (Spring Security). "%25" is disallowed in the URL path because it enables "double encoding" of other problematic characters (like ".."), see https://capec.mitre.org/data/definitions/120.html
The solution was to move the search term from the URL path to a URL parameter. In most cases, this was straightforward and was done so on non-public APIs. In three cases, a versioned, public API needed to be changed and so a new version is introduced:
Get Entities via Search Criteria (other parameters elided)
old: /qdc/entity/v1/getEntitiesByCrit/{targetFlds}/{searchTxt}
new: /qdc/entity/v2/getEntitiesByCrit/{targetFlds}?searchText=term
Get Fields by criteria (other parameters elided)
old: /qdc/field/v1/getFieldsByCrit/{targetFlds}/{searchTxt}
new: /qdc/field/v2/getFieldsByCrit/{targetFlds}?searchText=term
Get Sources by Criteria (other parameters elided)
old: /qdc/source/v1/getSourcesByCrit/{targetFlds}/{searchTxt}
new: /qdc/source/v2/getSourcesByCrit/{targetFlds}?searchText=term
All Searches Now Default to "Starts With" Matching Behavior
Jira ID: QDCB-1053
Several releases ago, the Catalog module search was changed from a "contains" search (SQL: like '%term%') to a "starts with" search (SQL: like 'term%'). This was done for performance reasons. A core_env property exists to enable the legacy "contains" behavior.
This release converts all of the other searches (e.g., global search at the top-right of all the other modules, source search, target search, etc.) to use the same behavior and the same override property:
# Flag to control if the searched term in Catalog will have a percent sign prepended to it implicitly.
# The term always has a trailing percent sign appended to it implicitly. If the value of this flag
# is set to false, the user must put the percent sign (%) themselves in the search term for the code
# to do wildcard search. Default: false
#prepend.leading.percent.sign.for.catalog.search=false
August 2021 Initial Release (4.11)
User Audit of Sense Data Connections No Longer Uses Domain Suffix
Jira ID: QDCB-1031
When an AD user logs in to Catalog, an audit of Qlik Sense determines the data connections to which the user has access. The name used for the audit was fully qualified (e.g., jdoe@company.com). Sense security is managed using just the user name prefix (e.g., jdoe) and not the domain suffix. The audit code has been changed to only use the user name prefix. This may have prevented any QVDs from being diplayed for the user as no data connections may have matched the fully qualified username.
SAML Authentication Request Signing
Jira ID: QDCB-1043
When SAML authentication is used, if the Identity Provider metadata XML has element IDPSSODescriptor with WantAuthnRequestsSigned="true", a key-pair is required to sign requests (private key is used to sign, public key is shared in Service Provider metadata XML).
The code to do this is guarded by a new core_env property, "saml.sign.requests.using.tomcat.ssl.cert". Set it to true to enable SAML request signing if required by the IdP and if/when the following is seen: "java.lang.IllegalArgumentException: Failed to resolve any signing credential".
The Tomcat SSL key-pair from conf/server.xml will be re-used for this purpose.
Automatic Update of core_env Property "doc.webhelp.uri"
Jira ID: QDCB-1021
Previously, on upgrade, core_env property "doc.webhelp.uri" had to be manually updated to the latest doc release (e.g., https://help.qlik.com/en-US/catalog/August2021/Content/QlikCatalog/Overview/Home.htm). This is now done automatically.
NextGen XML Processing Fixed
Jira ID: QDCB-1045
A defect was introduced in the May 2021 SR1 release preventing XML entities from being defined when using the "NextGen XML" feature (core_env property enable.new.xml.ingestion=true). An HTTP 400 response was returned to the private API call /discovery/extractMetadataFromXSDFile. This has been fixed.
Open Issues
Issues Running Prepare Dataflows Using Tez Execution Engine on CDP 7
Jira ID: QDCB-1046
On multi-node Catalog deployments using CDP Private Cloud 7, Prepare Dataflows may encounter the following issues when using the TEZ execution engine:
(1) 5+ minute job completion. While the job itself runs quickly, the handshake between Pig and Tez over job completion takes 5+ minutes. This overhead means Tez is not suitable for smaller Dataflows.
(2) coop.error.code.GRAPH_EXECUTION_ERROR. Pig and Tez may not agree on object serialization formats.
The workaround is to use the LOCAL execution engine for small Dataflows, and the MAP_REDUCE execution engine for larger Dataflows.
Note that Prepare uses Pig, which is not supported on CDP 7. Catalog instead includes and uses an older version of Pig.
Upgrade notes
Migrating to Tomcat 9
Beginning with the May 2021 release, only Apache Tomcat 9 is supported, and the installer will prohibit other versions. If using Tomcat 7, please first initiate a migration to Tomcat 9 before installing this release. Then, when installing, the upgrade option (-u) is NOT used.
|
Step
|
Sample Commands
|
|
Shutdown and rename old Tomcat 7
|
cd /usr/local/qdc (or cd /usr/local/podium)
./apache-tomcat-7.0.<VERSION>/bin/shutdown.sh
mv apache-tomcat-7.0.<VERSION> old-apache-tomcat
|
|
Download and expand Tomcat 9 - NOTE: adjust version 9.0.45 to use latest 9.0.x series
|
wget https://apache.claz.org/tomcat/tomcat-9/v9.0.45/bin/apache-tomcat-9.0.45.tar.gz
tar -xf apache-tomcat-9.0.45.tar.gz
rm apache-tomcat-9.0.45.tar.gz
|
|
Copy core_env.properties from old Tomcat 7 to new Tomcat 9
|
cp old-apache-tomcat/conf/core_env.properties apache-tomcat-9.0.45/conf/
|
|
Extract server.xml from podium.zip and copy to new Tomcat
|
unzip -j podium-4.10-<BUILD>.zip podium/config/tomcat9-server.xml -d .
mv ./tomcat9-server.xml apache-tomcat-9.0.45/conf/server.xml
|
|
Configure QDCinstaller.properties for Tomcat 9
|
Whether using an existing QDCinstaller.properties file from a previous install, or configuring one for the first time, ensure that it is updated to point to Tomcat 9:
TOMCAT_HOME=/usr/local/podium/apache-tomcat-9.0.45
|
|
Finally, run the installer. Do NOT specify upgrade mode (-u), as some files should be created as if it were a first-time install.
|
./QDCinstaller.sh
|
At this point, Tomcat 9, when started at the end of installation, will support only HTTP on port 8080.
Verify successful Qlik Catalog startup and basic functionality.
Additional configuration will be required to configure HTTPS on port 8443, apply security headers, etc. If Tomcat 7 used HTTPS, the keystore (jks file) containing the public-private keypair should be copied to Tomcat 9.
In addition, Tomcat 7 may have been configured as a service. It should be disabled. Tomcat 9 may be configured as a service to automatically start.
Please see the install guide for guidance on both of these.
New Upgrade Process
Do not attempt to upgrade until the following is understood.
If upgrading from a version of Qlik Catalog prior to September 2020 there are utilities that MUST be run after Catalog is upgraded. Once run, the utilities need never be run again.
The server may not start until the first two utilities have been run and will log a WARN at startup until the third is run. Do NOT upgrade the server until familiar with these utilities and the information required to run them. It will take time to gather this information. Gathering the information BEFORE Catalog is upgraded will minimize downtime.
Run the utilities in this order:
- jwt2CertsUtility -- please review readme.txt
This will be required if Qlik Sense Connectors have been defined in order to load QVDs.
Will need to gather networking info and certificate files from Qlik Sense servers.
May be run from any directory.
- singleNodeUpgradeForEntitiesWithBadOrUglyData.sh -- please review comment in script
This will be required if the installation is single-node.
Will need podium_dist database info if defaults altered.
May be run from any directory.
- singleNodeUpgradeToGrantReadOnlyUserAccessToDistSchemas.sh -- please review comment in script
This will be required if the installation is single-node.
Will need podium_dist database info if defaults altered.
May be run from any directory.
Downloads
Qlik Catalog August 2021 SR2 - Application
Qlik Catalog August 2021 SR2 - Installer
About Qlik
Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.
qlik.com
.png "Former Employee"){kind=icon}