Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik and ServiceNow Partner to Bring Trusted Enterprise Context into AI-Powered Workflows. Learn More!
cancel
Showing results for 
Search instead for 
Did you mean: 
Peony
Creator III
Creator III

How to avoid data duplication when unpivoting data using concatenate

Hi all. Please advise how to avoid data duplication for the example below?
I have 2 tables

tab1:
Load
    calendar_key,

    product_key,
    operator_code_1,
    operator_code_2
    sales_dollar,
    sales_unit
from Sales;

tab2:
Load
    operator_key,
    operator_Id,
    operator_name
from Operator;

I need to connect these tables using condition that operator_key = operator_code_1 and operator_key = operator_code_2.

My idea is to solve it like this

Tab_Fact:
Load
    calendar_key,

    product_key,
    operator_code_1 as operator_key,
    sales_dollar,
    sales_unit
from Sales;

 Concatenate(Tab_Fact)
Load
    calendar_key,
    product_key,
    operator_code_2 as operator_key,
    sales_dollar,
    sales_unit
from Sales;


But as result of this solution makes data duplication. And I'm struggling to find solution that will allow to avoid it. 
Also, just map operator_key to operator_code_1 and operator_code_2 is not an option. Operator_Id field should be used as single fields for the section access. 

I'll appreciate any ideas on this case. 


Labels (2)
2 Replies
Or
MVP
MVP

What you're describing is inherently going to duplicate data. This isn't a technical issue, it's a logical issue. You need to apply a logical solution to decide how you want to avoid duplication in this scenario, and then you (or people here) can help figure out the code aspect.

That said, if your entire issue is with Section Access, you can just leave the separated Operator fields and add a composite security key. Have a look at https://community.qlik.com/t5/Design/Basics-for-complex-authorization/ba-p/1465872 or just ask your friendly neighborhood AI about composite section access keys. 

 

 

vighnesh_gawad
Partner - Creator II
Partner - Creator II

Hi @Peony , you can try below code. you will get operator_Id in main table without any duplicates and it will also check operator_key = operator_code_1 and operator_key = operator_code_2 in same table.

Operator_Map:
Mapping
Load
    operator_key,
    operator_Id
From Operator;

Tab_Fact:
Load
    calendar_key,
    product_key,
    // First try operator_code_1
    // If not found then try operator_code_2
    ApplyMap(
        'Operator_Map',
        operator_code_1,
        ApplyMap('Operator_Map', operator_code_2, 'Unknown')
    ) as operator_Id,
    sales_dollar,
    sales_unit
From Sales;

 

Regards, Vighnesh Gawad
Connect with me on LinkedIn | GitHub