Skip to main content

Qlik

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
Anand_Rao
Employee
Employee
‎2024-03-13 03:32 PM

March Ahead: Qlik Cloud Data Integration's Latest Innovations

Buckle up, data wizards and dashboard divas! Our March release continues to deliver features so hot that it’ll melt the winter snow off your driveway. This month, we’ve focused on capabilities to enhance your data management user experience. So, let’s dive right in!

Data Movement Gateway Improvements

As part of our continued commitment to ensure access to data is as seamless as possible, we have made a bunch of improvements to the Data Movement Gateway to offer greater flexibility in deployment and supporting new sources and targets, database versions, and driver versions.

Deployment Options

It is now possible to install the Data Movement gateway on Red Hat 9.x* or on any corresponding and compatible Linux distribution. In addition to assisting in managing deployments a new notification feature has been introduced that will notify admins when a new gateway version is available.

Anand_Rao_0-1710356933895.png

 

New Data Sources and Target data platforms

We have really strong strategic technical and commercial partnerships with the technology vendors and we are continually improving our support of data movement options for databases and cloud data platforms.

Source Updates

  • Azure Database for MySQL - Flexible Server – we have added this new data source which is supported via the existing MySQL source connector.
  • MariaDB versions – we have expanded the support for MariaDB versions from 10.4 to 10.11 (previously 10.4 and 10.5)

Target Updates

  • Azure Database for MySQL - Flexible Server – we have added this new target which is supported via the existing MySQL target connector.
  • Databricks (Cloud Storage) – we have expanded our support to include Databricks 13.3 LTS and Serverless SQL Warehouse.

 

*Note: If you have an SAP HANA source that you want to use with a Data Movement gateway on Red Hat Linux 9.x, you must ensure you are using SAP HANA ODBC 64-bit Driver version 2.0.19 or later.

For information about how to upgrade to the latest version, see Upgrading the Data Movement gateway installation.

New Monitor Views

Monitor Views provide more granular visibility into task operations with the ability to create monitoring perspectives via various filtering options. This release introduces two new filters based on “Source Type” and “Data Movement Gateway”:

Anand_Rao_1-1710357031719.png

Source Type Filter

This filter creates monitor views that only show tasks for specific source technologies. With this new filter you can now focus on tasks with specific sources of interest, e.g. your tasks for SAP Landing or your SQL Server Replications. 

Data Movement Gateway Filter

This filter creates a monitor view that lists all tasks for a specific gateway to simplify administration and management. It is really handy when an organization operates multiple gateways. For example, use the view to stop all tasks on a specific gateway before upgrading. Then, restart all gateway tasks once the upgrade is complete to ensure everything is operational. 

Additional Data Landing Configurations

Qlik Cloud Data Integration landing tasks are the first step in transferring data into your cloud data warehouse. This release provides additional configuration options to control the behavior of the landing task and improve performance.

 Anand_Rao_2-1710357159615.png

Control Tables

Control tables store additional operational information in the target data platform, such as landing status, suspended tables, and task history (including the number and volume of records processed during landing).

Error Handling Policies

Error handling policies allow you to control what happens when data truncation or other data errors occur. Choices include ignoring errors, suspending table updates, or stopping the task.

Data Preview

The new data preview feature displays the live data for the currently active task without leaving the Qlik Cloud Data Integration interface. The preview helps validate and shape the data while designing pipelines. 

The data preview is currently available for physical tables and SQL transformations. 

Physical tables behind logical datasets.

This shows the actual schema and data in the physical tables that logical datasets are mapped to.

Anand_Rao_3-1710357239560.png

Anand_Rao_0-1710357957999.png

 

Output of SQL transformations.

This allows previewing the result set of a SQL transformation while writing the query, even before any physical tables have been created. This is more powerful than just previewing source data.

 

Anand_Rao_3-1710358011395.png

 

Security Controls:

  • Can be disabled at the tenant level completely.
  • Can be enabled/disabled at the space level (currently same as project level). This allows enabling for dev/staging spaces but disabling for production spaces.
  • Can be restricted at the user level - to allow only certain users to view data.

 

So, in summary, this new data preview feature is a huge benefit for users to validate data and shape transformations, with granular security controls to restrict access to production data. The initial phase focuses specifically on SQL transformations, providing strong value and thereby allowing data shaping entirely within Qlik Cloud Data Integration. Additional data preview capabilities will be rolled out progressively over 2024.

What Next?

All these updates are available from March 12th, 2024, check them out now in your Qlik Cloud Data Integration tenant. If you don't yet have access to Qlik Cloud, why not take a guided tour and experience the value it can bring by yourself here?

 

480 Views
Subscribe by Topic