Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I encourage you to check out the new web page, product documentation, and great video resources for more information. Finally, we know many of you have questions about this offering, so I'll devote the remainder of this article to the top 5 most frequently asked questions about Qlik Cloud Data Integration
1. What is Qlik Cloud Data Integration?
Qlik Cloud Data Integration is a powerful solution for data engineers tasked with deploying enterprise data integration and transformation initiatives. The offering forms a data fabric that delivers, transforms, and unifies your data across your organization via flexible, governed, and reusable data pipelines. These pipelines improve data timeliness, reliability, and scale, which are essential for every analytics, ML, or digital transformation initiative.
Consequently, Qlik Cloud Data Integration ensures that you can securely deliver the correct data (from on-premise or cloud sources) to the right destination (i.e. popular cloud data platforms), in the right format (with out of the box transformation features or custom SQL code), in near real-time.
2. Is Qlik Cloud Data Integration an entierely new offering
No. Qlik's journey to becoming a cloud business provided a unique opportunity for Qlik to rethink how we deliver data integration capabilities. But this wasn't the only factor. We also considered where the data integration market was going, what features our customers were requesting, and if there were opportunities we should pursue.
- Customer requirements – Customers have been assembling their own "data stacks" around the cloud data warehouse stores. The stacks are responsible for ingesting data, transforming data, cleansing data, protecting data, and serving data to various consumers. Our customers have told us repeatedly that managing all these different functions can be overwhelming.
- Market Drivers – The past few years have seen a dramatic rise in the adoption of cloud data platforms such as Snowflake, Databricks, Azure Synapse, Google Big Query, and Amazon Redshift. These platforms have become virtually every Enterprise's central data store for many downstream analytics initiatives.
- Qlik's Experience – Qlik's years of real-time data integration and transformation experience highlighted a gap that transforming transactional records into analytics-ready tables is a highly desirable and repeatedly performed activity. Most
Therefore we decided to create a new offering rather than simply porting existing client-managed solutions after analyzing all the data and research.
3. Where should I use Qlik Cloud Data Integration?
There are three common scenarios where Qlik Cloud Data integration is a great fit today:
- Real-time Data Warehouse Ingestion – You can use Qlik Cloud Data Integration to build a pipeline that continuously ingests data into a cloud data warehouse. Data can be ingested from on-premises data sources such as databases, SAP or mainframe applications in near real time. You can also load data from cloud applications too. The Illustration below demonstrates how two pipelines ingest data into Snowflake from an on-premise Oracle and MySQL database, while the third pipeline loads data from Salesforce.
- Automatic Star Schema Data Mart Generation – The second use case focuses on turning transactional records into analytics-optimized data marts. The beauty of Qlik Cloud Data Integration is that data mart generation is part of the data pipeline. Many market alternatives require the use of third-party data transformation tools, but Qlik doesn't. Qlik offers a visual data modeler, generates the necessary push-down SQL, and takes care of the execution scheduling too. The image below shows the graphical modeler automatically creating the dimension and fact tables within your data warehouse.
- Custom SQL for New Workloads – The last use case is creating pipelines for innovative workloads such as predictive analytics or machine learning. These workloads often require specialized data sets for training and prediction. With Qlik Cloud Data Integration, you can build real-time pipelines that combine data from various sources, filter and prepare the subsets, then deliver the result to the respective training and production environments.
4. Are there any changes to client-managed Qlik Data Integration?
No! Client-managed Qlik Data Integration is still alive and well and in active development. Furthermore, we have a regular release cadence and are committed to supporting our existing customers. We recognize that not all customer workloads can run in the cloud and we're not forcing anyone to migrate. Moreover, we've also made the two solutions interoperate without being integrated for those customers that want to run both on-premises and cloud workloads.
5. How can I try Qlik Cloud Data Integration?
Use this contact form link or call your account manager to get your hands on Qlik Cloud Data Integration today! Our data integration experts will help you make the most of this paid add-on to ensure that your first data integration project is a success.
Conclusion
I'll end the article where I began. I'm really excited that we announced Qlik Cloud Data Integration today, and how you can use it to create pipelines that deliver data and automate transformation in leading cloud data warehouses. However, what excites me most is the anticipation of hearing how the Qlik community and customers will embrace and deploy this fabulous new offering.
REGISTER for "DO MORE WITH QLIK" session on Nov 09 covering "Qlik Cloud Data Integration"
Clive Bearman
Sr Product Manager