Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Jan 20, 2022 10:34:24 AM
Apr 2, 2021 4:49:54 PM
Qlik Compose data warehouse projects provide end to end lifecycle
management of your data warehouse. The model is a focal point as it provides the foundation for much of the ETL and data mart automation capabilities in Qlik Compose.
This technical paper is a guide to understanding Qlik Data Integration warehouse modeling design techniques and best practices for common business scenarios.
Greate! Do you have a similar guide for data lakes?
Compose Data Lake projects are purpose built to manage a pure ODS ("replicated copy of data") and HDS ("historical view of data changes - ie. Type 2") that typically conform to the source system.
Consider a typical data lake architecture with 'RAW / Landed data' --> Bronze --> Silver --> Gold where the further "right" you go in the architecture, the more you implement transformation / curation of data. Compose Data Lake projects are built to automate the "Bronze" layer of your data lake environment.
Therefore there is no "modeling" concepts in Compose Data Lake like there is in Data Warehouse projects. Typical lake implementations in Compose DL projects are handled with Discovery of metadata (which also pre-generates mappings), generation of code and then execution (followed of course by deployment 🙂 ).
Hope that helps
Thanks for the detailed answer