I’m thrilled to write this installment of Qlik’s innovation blog because the new Qlik Talend Cloud features I’ve chosen to highlight are two of the capabilities I’ve been testing over the past few weeks. So, without any further ado let's dive into these exciting new capabilities!
Cross Project Pipelines
Since it’s inception, Qlik Talend Cloud pipelines have offered straightforward design metaphor. Often, you’d create a pipeline for a single data source that continually landed, merged and transformed data changes into a single target, such as a cloud data warehouse or lake. As time progressed the ability to add multiple data sources to a pipeline was introduced, and dedicated replication tasks with multiple targets followed a short time later.
However, many customers gave feedback that they’d like pipelines to be more modular, especially as projects became bigger and more complex. Modularity would not only increase component reusability, but also enable pipelines to be segregated by business domain. In addition, pipeline development would be more flexible while adhering to the best data-design practices.
Well, I’m happy to announce that “Cross Project Pipelines” are now generally available in all tenants. You can split complex pipelines consisting of multiple ingestion and transformation tasks into components that can be reused by other projects providing greater design flexibility and simplified pipeline management. In addition, Cross Project Pipelines can be segregated by data domain to encourage Business Domain Data Product or Data Mesh design principles.
AI Processor Snowflake Support
At the end of 2024, we released an AI processor that allowed you call native Databricks AI functions in a Transformation Flow without the need to hand code SQL. Databricks AI functions are a set of built-in SQL functions that allow you to apply AI directly to your data within SQL queries. This means you can use powerful AI models for tasks like sentiment analysis, text generation, and more, all from your Qlik Talend Cloud pipelines. If you can’t remember that far back then checkout this Qlik community blog post “Inject AI into your Databricks Qlik Talend Cloud Pipeline”
While many of our Databricks customers were overjoyed, the Snowflake proponents felt very left out, regularly commenting that Snowflake Cortex offered similar features too. Those comments were frequently followed by the question of “When will Qlik’s AI processor support Snowflake too?” Once again, I’m happy to say we’ve listened, and now the AI processor also supports Snowflake Cortex AI functions as well! The details of how to use Snowflake Cortex go beyond the scope of this blog post but stay tuned because a detailed article and demo of this feature will be published shortly. Until then, look at the screenshot below to see the AI processor in action and follow the link for more information about Snowflake Cortex LLM functions.
Wrap Up and 2025 Roadmap Webinar
Well there you have it. Two great new features that expand the usefulness and uses of Qlik Talend Cloud, but it doesn’t stop there. If you’re curious about what other innovations, enhancements, and improvements are coming to the Qlik platform in 2025 then join our Qlik Insider Webinar - Roadmap Edition that’s taking place on February 26th. Follow this link and register today!