Qlik Community

Knowledge

Search or browse our knowledge base to find answers to your questions ranging from account questions to troubleshooting error messages. The content is curated and updated by our global Support team

Announcements
QlikWorld online is next week! REGISTER NOW

Qlik Replicate: Many-to-One Replication Configuration

cancel
Showing results for 
Search instead for 
Did you mean: 
Michael_Litz
Support
Support

Qlik Replicate: Many-to-One Replication Configuration

Click here for Video Transcript

Environment

Overview

One popular configuration option for Replicate tasks is to handle a Many-to-One scenario. Imagine regional source sales databases that you want to consolidate into a single target database in order to simplify reporting needs across regions. This article will describe how to set up and configure replicate tasks to handle this situation, and introduce some key concepts for this to work.

Michael_Litz_0-1614629348609.png

Source Database

Lets start with the source databases.

  • Key Concept The source database tables structure must be identical across the many source database.

    Michael_Litz_1-1614631575682.png
  • Key Concept No guarantee that the primary key values on tables will be unique across the many source databases.
    Michael_Litz_2-1614631640296.png

Each source tables primary key will have its own unique key values. If these values are duplicates in other source. databases tables there will be conflicts in the target database as the many sources get replicated to the target
( There would be duplicate primary key issues). 

NOTE: This potential conflict will be taken care of in the task design/transformation

Task configuration

When configuring Replicate tasks to handle a Many to One scenario you will need to have a task for each source database. Each task will do Full Load and CDC  to the same target database thus creating the merged database that contains all the tables and values from each source in one target database. In my example I will only have one table in my tasks called customer.
My example has (Region_1 to Region_ALL) and  (Region_2 to Region_ALL)  tasks configured.

 

Michael_Litz_1-1614639972388.png

For each task you define there will be a unique source endpoint and the identical target endpoint.

There are 2 source endpoint and only one target endpoint defined, as each task will use the same target endpoint.

Michael_Litz_0-1614639918065.png

Key Concept Each task must be set to Do Nothing on the Full Load/Full Load Settings/Target Table Preparation screen. If the tables do not exist then the task will create them. This setting insures that as each task runs its full load the task will not drop or delete records from the previous task full load. NOTE: It is okay to set this during the task creation as the task will still create the tables if they do not already exist.

Michael_Litz_0-1614640214260.png

Key Concept Table level transformation adding a new field to the target side table that becomes part of the tables primary key. You can see the table transformation for the Customer table in this image.

Michael_Litz_1-1614640539868.png

The field RegionCode has been added to the target table Customer, then click on the "Key" column to add it to the PK on the target. For each task hard code a unique number for the region code. i.e. 1, 2, 3 etc.

NOTE: This does not impact the source table, it is target side only.

Target Database

Lets have a look at the target side database/tables. You can see that the RegionCode field has been added to the table and it is part of the PK, thus insuring there will be no duplicate key errors.

The customer table structure.

Michael_Litz_2-1614640772431.png

 

The customer table showing the merged records from both Region 1 and Region 2 source databases.

Michael_Litz_3-1614640885978.png

 

NOTE: Please also watch the accompanying video on how to configure this many to one environment.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution above may not be provided by Qlik Support.

Related Content

Tags (1)
Labels (2)
Version history
Last update:
‎2021-04-09 10:52 AM
Updated by: