Qlik Community

Qlik Compose for Data Warehouses

Discussion board for collaboration on Qlik Compose for Data Warehouses.

cancel
Showing results for 
Search instead for 
Did you mean: 
vtatarnikov
Creator
Creator

Reuse of mappings in Qlik Compose

I have several identical tables in the landing zone. I want to reuse one mapping created once. Tell me how to do it.

Labels (1)
1 Solution

Accepted Solutions
TimGarrod
Employee
Employee

Hello,   With the February Service Release of Compose (2021.8.0.465) , you can now "re-use a mapping" via the Uniform Source Consolidation feature.   

With this feature, you can assign multiple Landing Zones to a specific ETL Task and Compose will automatically generate the full or cdc load code to re-use the mapping logic, validations etc across the multiple sources. 

This feature does require that the source views / tables have the same set of columns / metadata and also provides the ability to combine mappings / sources within an ETL Task. 

For example - 

ETL Task #1 could use MappingA and MappingB for Landing/ Sources 1 & 2 (load data for both sources at the same time)

ETL Task #2 could use MappingC and MappingD for Landing /Source 1 (Use the mappingC/D logic for src 1 only)

ETL Task #3 could use MappingC and MappingD for Landing / Source 2 (Use the mappingC/D logic for src2 only). 

View solution in original post

9 Replies
TimGarrod
Employee
Employee

If you are saying - I have identical tables and I want them mapped to the same target you can simply leverage DUPLCIATE to duplicate an entire set of mappings (and point them to a different schema). 

 

In the ETL management screen - select the ETL Task/Set to duplicate.  (If you only want to duplicate N mappings instead of all, you can also create a specific ETL Task and assign the mappings you wish to duplicate). 

Click Duplicate - in the pop-up, select the appropriate "Landing Zone" and Schema and whether you want Full Load or CDC tasks created.  Compose will duplicate the mappings, setting the landing schema to the appropriate landing schema.   

 

TimGarrod_0-1634913652027.png

Note : This assumes the table names and columns are identical, just in different schemas.  If this is NOT your case (Source 1 has a table called TBL1, but Source 2 calls it TBLA - you can look at leveraging the CLI to export mapping CSV and adjust the mappings / rename them and then import to add the new mappings). 

 

Hope this helps!

 

vtatarnikov
Creator
Creator
Author

Thanks for the answer.
In the case you described, the task is duplicated, not the mapping. I want to use 1 mapping for several identical tables (in different schemes) within one task (I want to run one task to collect the same data from different servers into one common table in the storage).

jtompkins
Creator
Creator

TimGarrod
Employee
Employee

Using teh method I described - the Task and mappings will be duplicated.   You simply need to tie the new mapping to your existing task (and if not needed, delete the Task). 

The default behavior in Compose is to align a Task with a source aka landing db/schema.  However you can create tasks to combine mappings however you see fit.  If you enable multiple mappings to load to the same target table in a single Task - then the INSERT into TSTG will occur in parallel and subsequent ELT statements will execute against the entire data set (akin to having a "UNION ALL" mapping). 

TimGarrod
Employee
Employee

For example - I have the following mapping 

TimGarrod_0-1635273913673.png

I select to DUPLICATE the task - 

TimGarrod_1-1635273954372.png

 

I now have 2 mappings (and yes, 2 tasks) = but my mapping logic is duplicated and the source schema is now pointing to my second set of source tables - 

TimGarrod_2-1635274006176.png

Mapping - 

TimGarrod_3-1635274021187.png

I can simple assign the duplicate mapping to the original Task by checking the check box next to the mapping name (and delete the duplicate task if you wish).   

TimGarrod_4-1635274061950.png

Generate the code and review the task statements -  note the 2 "Populating staging table for customers" statements - they will run in parallel....

TimGarrod_5-1635274123605.png

 

 

vtatarnikov
Creator
Creator
Author

Thanks, I got the idea. But this looks like a workaround. And probably if you need to change the logic, you will have to change it in all mappings, instead of one template.

vtatarnikov
Creator
Creator
Author

Sorry, but I cannot call this a solution

TimGarrod
Employee
Employee

Hello,   With the February Service Release of Compose (2021.8.0.465) , you can now "re-use a mapping" via the Uniform Source Consolidation feature.   

With this feature, you can assign multiple Landing Zones to a specific ETL Task and Compose will automatically generate the full or cdc load code to re-use the mapping logic, validations etc across the multiple sources. 

This feature does require that the source views / tables have the same set of columns / metadata and also provides the ability to combine mappings / sources within an ETL Task. 

For example - 

ETL Task #1 could use MappingA and MappingB for Landing/ Sources 1 & 2 (load data for both sources at the same time)

ETL Task #2 could use MappingC and MappingD for Landing /Source 1 (Use the mappingC/D logic for src 1 only)

ETL Task #3 could use MappingC and MappingD for Landing / Source 2 (Use the mappingC/D logic for src2 only). 

vtatarnikov
Creator
Creator
Author

Hi Tim, great news! Thank you!