Qlik Community

Qlik Product Innovation Blog

Learn about what's new across all of the products in our growing Qlik product portfolio.

Now Live: Qlik Sense SaaS Simplified Authoring – Analytics Creation for Everyone: READ DETAILS

Our data integration enhancements include simplified data lake creation, improved data warehouse automation, and platform integrations. Changed data propagated to data lake landing zones can now be optionally viewed when current and transactionally consistent leading to accurate, real-time insights. We now add Databricks Delta Lakes on Google’s platform to our ever-expanding list of supported data lakes. With support for a significant number of entities, automated datamarts can now support more comprehensive analyses. A single user interface to track Compose workflows across lakes and warehouses eases management, while tight integration of ingestion processes from one or many Replicate data pipelines to transformation and provisioning processes automate and speed data delivery to consumers. Lastly, ingestion and transformation of semi-structured data within Snowflake bring in new data sources.

A new release of Qlik Data Integration is here with exciting updates to Compose and Enterprise Manager. These updates address our customers’ needs across data lake creation, improved warehouse automation, and platform component integrations.


Let us start with simplified data lake creation.

Transactionally Consistent Live Views for Optimal Latency. Data consumers want a way to get the freshest data while ensuring high accuracy, a challenge often encountered with constantly updated data.   Consumers now can increase data integrity by enforcing transactional consistency while viewing changed data ingested into data lakes. This option leads to more accurate real-time insights with minimal delays in the freshness of data.


Support for Databricks on GCP. Users adopting Databricks Delta lakes in Google Cloud environments wanted to continue using the management tools they have used in AWS and Azure environments. Users can now automate the creation, hydration, and changes to Delta Lakes on GCP using the latest Compose release.


Second, let us examine improvements to data warehouse automation.


More comprehensive data marts. As datamarts have grown, so have the entities within them. Users can now easily automate the lifecycle of these large data marts to support comprehensive analyses across a much larger set of entities using an improved user interface with minimal declines in performance.


Support for a variant data type in Snowflake. Users expanding their use of Snowflake data warehouses often look to incorporate semi-structured data. Now, users can ingest and transform semi-structured data within their automated Snowflake data warehouses. 


Finally, let us investigate data integration platform enhancements.


Workflow Support in Enterprise Manager. Users seek a single pane of glass to obtain a comprehensive view to initiate and track various automation and creation tasks. In addition, to Replicate tasks, users can now start and track Compose workflows across data lakes and warehouses using a single interface or a third-party tool of their choice that integrates with an expanded set of Qlik platform APIs.


Multiple Replicate Servers. Users automating their warehouses and lakes are looking to ingest data in real-time from numerous distributed source systems. Now users can onboard and process data faster into their data lakes or warehouses by scaling to multiple simultaneous real-time data pipelines and automating complex distributed data architectures.

View these capabilities in action in the video below using Databaricks on GCP as an example: 

Click here for the Transcript

Try out the release by going to Support > Downloads on Qlik.com and filtering your options to Qlik Compose for Data Lakes (or Data Warehouses) version 2021.8 and Qlik Enterprise Manager 2021.5 SR2.


Register for the upcoming Data Integration Roadmap Session on September 16 at 11am EST.   Register here