Skip to main content

Qlik

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Upcoming release cadence changes for Qlik Sense Enterprise Client Managed environments. Details shared here.

Qlik Catalog Release Notes - May 2021 Initial Release to Service Release 3

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
djenkins-qlik
Employee
Employee

Qlik Catalog Release Notes - May 2021 Initial Release to Service Release 3

Last Update:

Oct 14, 2022 2:36:36 AM

Updated By:

frida_tell

Created date:

Oct 13, 2022 3:31:19 PM

Table of Contents

 

The following release notes cover the versions of Qlik Catalog released in May 2021. 

 

What's new in Qlik Catalog May 2021 Service Release 3

  • QDC-1285 - Address log4j2 Zero-Day Exploits, CVE-2021-44228, CVE-2021-45046 and CVE-2021-45105

  • With the May 2021 release, Microsoft Internet Explorer 11 is no longer supported.

 
SAML Support

Single sign-on using SAML authentication has significantly changed. Please see the “Enabling SAML” section      in the installation guide.

 

Encrypted QVD Support

Qlik Catalog may now be configured to load encrypted QVDs using an engine running on a Microsoft          Windows Qlik Sense server. Please see the “Loading Encrypted QVDs” section in the Qlik Catalog + Qlik     Sense Integration Guide.

 

Logging

Logging configuration changes, for single-node installations, are now made in file WEB-INF/classes/log4j2.xml rather than log4j.xml. Formerly, a logger entry looked like:

   <logger name="com.nvs.ui.interceptor.LoggingInterceptor">

      <level value="info" />

   </logger>

 

Now, in log4j2.xml, it looks like:

   <Logger name="com.nvs.ui.interceptor.LoggingInterceptor" level="info" additivity="false">

      <AppenderRef ref="Console" />

   </Logger>

 

Hive JDBC Driver

The Hive JDBC driver is no longer included by default for single-node installations.

If you want to load from a Hive JDBC source, or publish to Hadoop+Hive, you will need to get the appropriate Hive JDBC driver for your cluster. The example below allows a single-node installation to interact with Hive running on a CDH 5.16.2 cluster.

Copy the JDBC driver jar(s) to a dedicated directory (also add the three log4j2 jars from WEB-INF/lib):

$ pwd

/usr/local/qdc/jdbcDrivers/hive-jdbc-cdh5.16.2

$ ls -la

total 51360

drwxr-xr-x  6 qdc  staff   204B Jan 29 17:25 .

drwxr-xr-x  7 qdc  staff   238B Jan 29 16:31 ..

-rw-r--r--  1 qdc  staff    23M Jan 29 16:30 hive-jdbc-1.1.0-cdh5.16.2-standalone.jar

-rw-r--r--  1 qdc  staff   285K Jan 29 17:21 log4j-api-2.13.2.jar

-rw-r--r--  1 qdc  staff   1.6M Jan 29 17:25 log4j-core-2.13.2.jar

-rw-r--r--  1 qdc  staff    23K Jan 29 17:18 log4j-slf4j-impl-2.13.2.jar

 

There is already a "HIVE" JDBC driver defined in the Admin “Database Drivers” section of the UI. It must be updated to point to the above directory:

update podium_core.pd_jdbc_source_info

    set alt_classpath='/usr/local/qdc/jdbcDrivers/hive-jdbc-cdh5.16.2' where sname='HIVE';

A Tomcat restart is required.

 

Resolved Defects

 

Qlik Catalog May 2021 SR3

 

Address log4j2 Zero-Day Exploits, CVE-2021-44228, CVE-2021-45046 and CVE-2021-45105

Jira ID: QDC-1285

Single-node Catalog has been upgraded to use log4j2 version 2.17.0.

 

Qlik Catalog May 2021 SR2

 

Fixes for Publish to Qlik Sense, Publishing to an Existing App

Jira ID: QDCB-1023

For Publish to Qlik Sense, when publishing to an existing app, registered non-QVD entities were not being "loaded on demand". This meant that new data in the source system (e.g., a relational database table) was          not seen in Qlik Sense. This issue has been addressed.

Also, when publishing to an existing app, a new LOAD or SELECT script is appended to the previous script     (this allows the user to add an entity to an app). For some scenarios (e.g., when publishing REGISTERED            or ADDRESSED entities), this required the user to hand-edit the script and remove a duplicated        LOAD/SELECT script.

Catalog can now be configured to automatically replace the previous script. Set the following core_env property   to true for this behavior:

# When publishing to an existing app, this property controls whether a new LOAD/SELECT script is appended to the end

# of the script, or if the previous script is replaced. If true, "//CATALOG-SCRIPT-START" and "//CATALOG-SCRIPT-END"

# will be included in the script, and used to demarcate the script to be replaced. Be careful if setting to true -- if

# you later add entities to the app, the previous entities will be overwritten unless moved above //CATALOG-SCRIPT-START

# or below //CATALOG-SCRIPT-END. Setting this to true is appropriate if publishing ADDRESSED or REGISTERED entities,

# using INTERNAL mode, as the publish to an existing app feature is used to force an on-demand load. Default: false

#podium.qlik.replace.previous.script=false

 

SAML2 Service Provider Endpoint Implemented

Jira ID: QDCB-1028

With the May 2021 release, Catalog upgraded the Spring Security SAML2 library. The service provider metadata endpoint (URI) now looks like:

https://catalog-host:8443/qdc/saml2/service-provider-metadata/catalog

However, Catalog returned a 404 (not found) when this was accessed. This issue has been fixed. Note that the above endpoint was not necessary for Identity Providers (IdPs) such as Okta and MS Azure AD.

 

Newly Created Custom Properties Now Visible After Log Out/In

Jira ID: QDCB-1029

When using SAML authentication, when a user with ADMIN role added a property definition, then logged out and logged in, the newly added property was not available to add to an Entity or Source. The Catalog user interface has been fixed to always [re]load property definitions on login.

 

Fixed the search wildcard when source filter is selected

Jira ID: QDCB-1030

Search dataset work properly with wildcard

 

Qlik Catalog May 2021 SR1

 

Allow Ticket Acquisition for Sense URL to be Skipped

Jira ID: QDCB-964

At a high level, when Publish-to-Qlik-Sense occurs, the Catalog backend: (1) creates an app; and (2) acquires a user ticket for the UI redirect to Sense. Acquisition of the user ticket may be unnecessary (when Sense uses SSO authentication) and may be problematic if the user ticket acquisition URL is proxied through an appliance that interferes with authentication.

Acquisition of the user ticket may now be skipped. The Sense redirect URL will no longer have the "qlikTicket" parameter appended to it. A new core_env property is added -- set this property to true to skip "qlikTicket" acquisition and usage.

# If true, then Catalog will _NOT_ call Sense to generate a user ticket and will _NOT_ append a ticket to either

# qlik.sense.url or qlik.sense.redirect.url. One scenario where this would be used: if qlik.sense.redirect.url is used

# and represents a single sign-on (SSO) end-point. In that case, ticket-based authentication would not be used.

# Default: false

#qlik.sense.skip.ticket=false

 

Issue Fixed with "Add Field" Feature

Jira ID: QDCB-975

When using the "Add Field" button to add a Field to an Entity, for the single-node product, a backend error occurred. This issue has been fixed.

 

Retain "Load Data" Checkbox for Recurring QVD Import

Jira ID: QDCB-1006

When scheduling a recurring QVD Import, the selection of the "Load Data" checkbox was not saved. This setting is now correctly saved.

 

Redshift Details Now Tested when Defining Publish Target of Type "Amazon - S3 and Redshift"

Jira ID: QDCB-1008

When defining a Publish Target of type "Amazon - S3 and Redshift", the "Test Connection" button only validated the S3 details. Testing the connection failed to validate the Redshift details, allowing incorrect data to be saved for URI, username and password. Now, Redshift details are correctly validated.

 

Active Directory Support Enhanced to Enable Child Domain Logon

Jira ID: QDCB-1009

A customer requires all Catalog users imported from an Active Directory forest to share a common domain suffix in their logon name (e.g., jdoe@acme.com). The actual authentication must occur against the user's child domain (e.g., jdoe@europe.acme.com). A new core_env property "extract.realm.from.distinguished.name" enables       this scenario.

In core_env.properties, it is described as:

# If true, the realm (aka domain name) will be extracted from the user's distinguished name (DN) that

# was sync'ed from Active Directory using LDAP. This allows a global user name of "jdoe@acme.com"                 # to be entered at the login page, but the authentication to occur against a child domain using a name                  # of "jdoe@europe.acme.com". One way to configure this scenario is as follows:

# - set core_env property ad.use.alias.as.domain.name=true

# - in Active Directory, place all Catalog users, in all domains, in a universal group

# - as an admin in the Catalog UI, name the domain alias "acme.com"

# - set the LDAP group query to filter on the universal group, (cn=universalGroupName)

# - set the domain host to be the Global Catalog server and port (typically 3268)

# - the DN for each user will be captured during AD sync, it will look something like                                                     #   CN=jdoe,OU=AcmeProducts,DC=europe,DC=acme,DC=com

# - when the user logs in to Catalog, their Catalog user name is "jdoe@acme.com"

# - however, Catalog will use "jdoe@europe.acme.com" as the Kerberos principal when authenticating the user

#

# When this property is set to true and the user has a DN, properties java.security.krb5.realm and

# another.realm.username.pattern are ignored. Default: false.

#extract.realm.from.distinguished.name=false

 

Support for iSeries (AS/400) JDBC Loads Confirmed

Jira ID: QDCB-1020

Basic smoke testing of JDBC loads from an iSeries (AS/400) instance have been confirmed. Full regression testing was not completed.

Driver details:

  • the driver can be downloaded from http://jt400.sourceforge.net/
  • driver version 10.6 was verified
  • copy java8/jt400.jar to the directory identified by core_env property "jdbc.alternate.classpath.dir"                                           (e.g., /usr/local/qdc/jdbcDrivers)
  • restart Tomcat

With the May 2021 SR1 (4.10.1) release, an "AS400" Database Driver entry is automatically created. In an older release, to manually create it:

  • in the UI, use Admin / Database Drivers / Add Driver to add a new driver entry, name it something like "AS400"
  • the driver class name is "com.ibm.as400.access.AS400JDBCDriver"
  • the start/end character is double-quote "
  • the sample URL is jdbc:as400://hostname/default-schema

Once the driver has been defined, create a new Source Connection of type JDBC and using the AS400 driver.

 

Qlik Catalog May 2021

 

Specifying Additional Security-Focused HTTP Response Headers

Jira ID: QDCB-941

Additional security-focused HTTP response headers may now be specified in core_env.properties, using

property "additional.response.headers" -- see example content below.

If header "Clear-Site-Data" is specified, it will only be included with a response to a request for /logout

or /logged-out.

# Additional security-focused HTTP response headers, pipe (|) separated. If header Clear-Site-Data is specified,

# it will only be included with a response to a request for /logout or /logged-out. Requires restart. Default: [none]

additional.response.headers=X-Download-Options: noopen|X-Permitted-Cross-Domain-Policies: none|Pragma: no-cache|Clear-Site-Data: "cache", "cookies", "storage"

 

Licenses container does not route to license.qlikcloud.com when OS-level 443 redirect is implemented

Jira ID: QDCB-954

In cases where an OS-level firewall on the Qlik Catalog node is configured to redirect traffic from 443 to 8443 to avoid specifying the port number in the Catalog URL, the license validation request does not route to https://license.qlikcloud.com. Instead, the request exits the licenses container subnet and is immediately redirected to the localhost port 8443.

Beginning with Catalog 4.10, docker-compose-qlikContainer.yml will set the docker container subnet for Catalog containers to 172.20.0.0/16:

   networks:

        qdc:

            driver: bridge

            ipam:

                config:

                    - subnet: 172.20.0.0/16

(This subnet range may be modified in the event of a conflicting subnet within a customer's network topology.)

Additionally, the following two firewalld rules should be created to allow the Catalog containers to route requests on port 443 appropriately:

443 Redirection to 8443:

firewall-cmd --permanent --direct --add-rule ipv4 nat PREROUTING 1 -p tcp --dport 443 -j REDIRECT --to-ports 8443

 

Container bypass port 443 redirection:

firewall-cmd --permanent --direct --add-rule ipv4 nat PREROUTING 0 -s 172.20.0.0/16 -j ACCEPT

 

Source Name "public" is Reserved

Jira ID: QDCB-955

The Source name "public" is reserved. This means that a JDBC schema named "public" can not be imported and loaded without taking action. During the JDBC Source and Entity definition process, the Source should be named to not conflict with "public", perhaps adding a unique suffix. There was no code change for this issue.

 

Support Publish to Qlik Sense for Users from Different Directories

Jira ID: QDCB-958

A customer may have imported users from multiple directories (domains) into both Qlik Sense and Qlik Catalog. When "Publish to Qlik Sense" is selected, apps are listed and created/updated using the logged-in Catalog user (or the user specified in core_env property 'podium.qlik.username'). However, until this release, the property 'qlik.sense.active.directory.name' only supported specifying a single domain. Now, multiple domains may be specified. Catalog will iterate through the combinations of domain-and-user until a combination is found that has previously logged into Qlik Sense. This matched user will then be used to list and create/update apps. If multiple domains are specified, two additional properties are needed.

The revised core_env property and two additional properties are:

# Directory name (aka domain name) which is combined with either the logged

# in user name or property 'podium.qlik.username'. Qlik Sense applications are

# then listed and created/updated using the user represented by combining the

# directory and user names. This user should have previously logged in to Qlik

# Sense. Required property when 'is.publish.to.qlik.enabled' is set to true.

# Multiple directories may be specified by using commas to separate each. In this

# case, Qlik Sense will be queried for each directory\user combination -- if that

# user previously logged in, that directory\user combination will be used for

# Publish to Qlik operations.

#qlik.sense.active.directory.name=

 

# If multiple directories were specified in property 'qlik.sense.active.directory.name',

# then a Qlik Sense 'RootAdmin' user is required in order to query Qlik Sense. Enter the

# directory and user names of a 'RootAdmin' user.

#qlik.sense.root.admin.directory.name=

#qlik.sense.root.admin.user.name=

 

Catalog Text Search Performance Improvement

Jira ID: QDCB-963

The primary database view (pd_dashbrd_with_fld_tag_v) supporting text search in the Catalog module (the product home page) has been restructured.

Previously, performance degraded as the number of search matches increased. A search match returning 10,000+ entities (paged in increments of 100) was executing in approximately 30 seconds.

With the restructured view, this same search now executes in less than 3 seconds (as measured at the server, not inclusive of network transport or browser rendering).

 

Fix for Business Metadata Tags Concatenated when XLS Format Used

Jira ID: QDCB-966

When importing Business Metadata, multiple tags should appear as distinct, separate tags in the Catalog UI and database. However, due to a defect, the discrete tags were concatenated as a single string, when the file format was XLS. This issue has been fixed. The issue was not present with file format CSV.

 

API Query for UI Security Group Wizard Optimized

Jira ID: QDCB-988

Customer reports server out-of-memory or UI hang when attempting to create Security Group with all Sources and Entities (28 Sources, 14,000+ Entities).

Both are related to a poorly structured query producing a large result set, large enough that either the server exhausted heap memory, or the UI could not process the resulting data.

The backend query has been optimized, as well as what is returned. For the customer dataset, 11.6 MB is now returned rather than 2.3 GB.

The entire round trip to the server + display takes about 35 seconds on a development server, which is long, but is no longer a hang. Roughly 15 seconds are spent at the server generating the result, and 20 seconds is spent by the UI processing and displaying the result.

 

Upgrade notes

 

Migrating to Tomcat 9

Beginning with the May 2021 release, only Apache Tomcat 9 is supported, and the installer will prohibit other versions. If using Tomcat 7, please first initiate a migration to Tomcat 9 before installing this release. Then, when installing, the upgrade option (-u) is NOT used.

Step

Sample Commands

Shutdown and rename old Tomcat 7

cd /usr/local/qdc (or cd /usr/local/podium)

./apache-tomcat-7.0.<VERSION>/bin/shutdown.sh

mv apache-tomcat-7.0.<VERSION> old-apache-tomcat

Download and expand Tomcat 9

wget https://apache.claz.org/tomcat/tomcat-9/v9.0.45/bin/apache-tomcat-9.0.45.tar.gz

tar -xf apache-tomcat-9.0.45.tar.gz

rm apache-tomcat-9.0.45.tar.gz

Copy core_env.properties from old Tomcat 7 to new Tomcat 9

cp old-apache-tomcat/conf/core_env.properties apache-tomcat-9.0.45/conf/

Extract server.xml from podium.zip and copy to new Tomcat

unzip -j podium-4.10-<BUILD>.zip podium/config/tomcat9-server.xml -d .

mv ./tomcat9-server.xml apache-tomcat-9.0.45/conf/server.xml

Configure QDCinstaller.properties for Tomcat 9

Whether using an existing QDCinstaller.properties file from a previous install, or configuring one for the first time, ensure that it is updated to point to Tomcat 9:

TOMCAT_HOME=/usr/local/podium/apache-tomcat-9.0.45

Finally, run the installer. Do NOT specify upgrade mode (-u), as some files should be created as if it were a first-time install.

./QDCinstaller.sh

At this point, Tomcat 9, when started at the end of installation, will support only HTTP on port 8080.

Verify successful Qlik Catalog startup and basic functionality.

Additional configuration will be required to configure HTTPS on port 8443, apply security headers, etc. If Tomcat 7 used HTTPS, the public-private keypair should be copied to Tomcat 9.

In addition, Tomcat 7 may have been configured as a service. It should be disabled. Tomcat 9 may be configured as a service to automatically start.

Please see the install guide for guidance on both of these.

 

New Upgrade Process

Do not attempt to upgrade until the following is understood.


If upgrading from a version of Qlik Catalog prior to September 2020 there are utilities that MUST be run after Catalog is upgraded. Once run, the utilities need never be run again.

The server may not start until the first two utilities have been run and will log a WARN at startup until the third is run. Do NOT upgrade the server until familiar with these utilities and the information required to run them. It will take time to gather this information. Gathering the information BEFORE Catalog is upgraded will minimize downtime.

Run the utilities in this order:

  1. jwt2CertsUtility -- please review readme.txt

This will be required if Qlik Sense Connectors have been defined in order to load QVDs.

Will need to gather networking info and certificate files from Qlik Sense servers.

May be run from any directory.

 

  1. singleNodeUpgradeForEntitiesWithBadOrUglyData.sh -- please review comment in script

This will be required if the installation is single-node.

Will need podium_dist database info if defaults altered.

May be run from any directory.

 

  1. singleNodeUpgradeToGrantReadOnlyUserAccessToDistSchemas.sh -- please review comment in script

This will be required if the installation is single-node.

Will need podium_dist database info if defaults altered.

May be run from any directory.

 

Downloads

 

Qlik Catalog May 2021 SR3 - Application
Qlik Catalog May 2021 SR3 - Installer

 

 

 

 

 

 

About Qlik

Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. A private SaaS company, Qlik offers an Active Intelligence platform, delivering end-to-end, real-time data integration and analytics cloud solutions to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.

qlik.com

Labels (1)
Labels:
287 Views
0 Likes
Release details
Products:
Qlik Catalog
Release name:
May 2021
Patch:
3
Language:
English
Version history
Last update:
‎2022-10-14 02:36 AM
Updated by:
Former Employee