Reuse of mappings in Qlik Compose

[vtatarnikov]

I have several identical tables in the landing zone. I want to reuse one mapping created once. Tell me how to do it.

[TimGarrod]

If you are saying - I have identical tables and I want them mapped to the same target you can simply leverage DUPLCIATE to duplicate an entire set of mappings (and point them to a different schema). 

In the ETL management screen - select the ETL Task/Set to duplicate.  (If you only want to duplicate N mappings instead of all, you can also create a specific ETL Task and assign the mappings you wish to duplicate). 

Click Duplicate - in the pop-up, select the appropriate "Landing Zone" and Schema and whether you want Full Load or CDC tasks created.  Compose will duplicate the mappings, setting the landing schema to the appropriate landing schema.   

Note : This assumes the table names and columns are identical, just in different schemas.  If this is NOT your case (Source 1 has a table called TBL1, but Source 2 calls it TBLA - you can look at leveraging the CLI to export mapping CSV and adjust the mappings / rename them and then import to add the new mappings). 

Hope this helps!

[vtatarnikov]

Thanks for the answer.
In the case you described, the task is duplicated, not the mapping. I want to use 1 mapping for several identical tables (in different schemes) within one task (I want to run one task to collect the same data from different servers into one common table in the storage).

[jtompkins]

I have an idea open for that. Ability to copy mapping within the same ETL set in Compose 

[TimGarrod]

Using teh method I described - the Task and mappings will be duplicated.   You simply need to tie the new mapping to your existing task (and if not needed, delete the Task). 

The default behavior in Compose is to align a Task with a source aka landing db/schema.  However you can create tasks to combine mappings however you see fit.  If you enable multiple mappings to load to the same target table in a single Task - then the INSERT into TSTG will occur in parallel and subsequent ELT statements will execute against the entire data set (akin to having a "UNION ALL" mapping). 

[TimGarrod]

For example - I have the following mapping 

I select to DUPLICATE the task - 

I now have 2 mappings (and yes, 2 tasks) = but my mapping logic is duplicated and the source schema is now pointing to my second set of source tables - 

Mapping - 

I can simple assign the duplicate mapping to the original Task by checking the check box next to the mapping name (and delete the duplicate task if you wish).   

Generate the code and review the task statements -  note the 2 "Populating staging table for customers" statements - they will run in parallel....

- 

[vtatarnikov]

Thanks, I got the idea. But this looks like a workaround. And probably if you need to change the logic, you will have to change it in all mappings, instead of one template.

[vtatarnikov]

Sorry, but I cannot call this a solution

[TimGarrod · Accepted solution]

Hello,   With the February Service Release of Compose (2021.8.0.465) , you can now "re-use a mapping" via the Uniform Source Consolidation feature.   

With this feature, you can assign multiple Landing Zones to a specific ETL Task and Compose will automatically generate the full or cdc load code to re-use the mapping logic, validations etc across the multiple sources. 

This feature does require that the source views / tables have the same set of columns / metadata and also provides the ability to combine mappings / sources within an ETL Task. 

For example - 

ETL Task #1 could use MappingA and MappingB for Landing/ Sources 1 & 2 (load data for both sources at the same time)

ETL Task #2 could use MappingC and MappingD for Landing /Source 1 (Use the mappingC/D logic for src 1 only)

ETL Task #3 could use MappingC and MappingD for Landing / Source 2 (Use the mappingC/D logic for src2 only). 

[vtatarnikov]

Hi Tim, great news! Thank you!