Modeling Real-time Data Warehouses in Compose

[TimGarrod]

In a modern analytics environment data warehouses must deliver more data than ever before to support right-time analytics and insights. For some enterprises this means reducing the end of day processing cycle, while for others it means delivering data in near real-time streams. However, the key to achieving this agility is to quickly harness complex data transformations that rapidly deliver the right data sets, at the right latencies, to the right people, at the right time.

Qlik Data Integration helps data architects simplify their transformation requirements to provide the flexibility to support multiple data velocities in near real-time for various data warehouse architectures.

This technical paper is a guide to understanding Qlik Data Integration warehouse modeling and transformation techniques and their impact on near real-time Change Data Capture (CDC) processing in an agile data warehouse environment.  

[clarcombe]

Very interesting @TimGarrod . 

I am currently in the process of creating ETL processes to load a data vault and then from the DV loading into a Data Mart on PostgreSQL

The DataVault is insert only (i.e. new rows supercede old rows but old rows are neverdeleted) using CDC from the data source

I'm wondering whether to abandon the PostgreSQL datamart and create it directly on Qliksense. Do you have any further reading/viewing which could help me with my decision please ?

Thanks

[TimGarrod]

Hi @clarcombe  - the white paper is related to Qlik's Data Integration capabilities - specifically Qlik Compose for Data Warehouses - which provides data warehouse automation.  This encompasses model assistance and ETL automation for a number of dw platforms. 

At this time however it does not support Postgres.    (I realize your question is around QlikSense - just wanted to give context to the white paper).   

I would ask that question in the forum for Qlik Data Analytics where some of our analytics experts will be able to weigh in.    

[clarcombe]

Thanks @TimGarrod 

I've looked at our data model and realised that I could probably get the data directly from DB2 instead of going through the data vault model (which I would have to build with my ETL)

Where can I find our more about the Qlik Compose ? Are there any learning modules I could look at ?

Thanks again