The Qlik Data Integration pipeline is powerful yet easy to implement to enable a complete Data Warehouse solution with minimal customization. The automation, both Replicate for data movement and Compose for data transformation to an analytics ready presentation layer, simplifies the tasks and effort require to deploy and operate the Data Warehouse environment.
In addition, Compose projects themselves can be further linked or chained to achieve enhanced flexibility of the resulting pipeline. This would be the case if for example we are looking to implement a super set Primary Data Warehouse (akin to a relational data lake) as first step with a Secondary Data Warehouse using a subset of the Primary as the second step. Another example would be implementing a subject area specific Data Warehouse, with very granular level of detail about its respective area, as a first step and Secondary DW that consolidates multiple subject area specific Data Warehouses at a higher level with less detail per subject. This flexibility requires task customizations and tighter dependencies between Compose projects and should be carefully evaluated against the status quo option of a single Compose project.
This paper will describe the considerations needed to implement the scenarios specified and how layering the architecture using multiple Compose projects solves for each of them.
