Defining Data Models and Mappings for Pivoting Data in Qlik Compose for Data Warehouses
Operational systems of record often provide data that is not readily consumable by analytics projects and frequently require complex extract, transform and load (ETL) processing to conform the information into a standard data warehouse model. One common transformation is to pivot multiple rows into a single row with multiple columns to provide a flattened data set for analysis. Most ETL technologies provide a capability to perform that transformation when processing data in batch; however, the Qlik Data Integration platform provides methods that support both batch and near real-time pivoting of source data into the data warehouse.
This paper provides real-world examples that explain the purpose of a pivoted data set, describes how to model the data structures, and how to implement transformation logic for batch and near real-time processing.