How to Delete Duplicates using tDenomalize and not just Merge same value.

MAni5 — Wed, 19 Mar 2025 21:58:43 GMT

We have duplicate values and we used tDenormalize to have a 1 row. However, the values are now on the single field with duplicate values and when we tick the merge same values. The generated output only merges the duplicate values where we would like to delete those and retain the unique data. We would like to seek the expertise of these community if there are solutions/components that functions the same way as what we expect. Please see below for the scenario.

Talend studio design component

Scenario

We wanted to delete the duplicate and not only merge them. For example, the value below before processing through tdenormalize

recordnumber	productname	email
2450	Creamer	Jessnie@email.ui
2450	Melona Banana	null
2450	Creamer	null

This will be process in tdenormalize the output would be the following
recordnumber	productname	email
2450	Creamer;Melona;Creamer	Jessnie@email.ui;null;null

When the merge same values is tick the output would be:
recordnumber	productname	email
2450	Creamer;Melona	Jessnie@email.ui;null


However, we wanted to delete those duplicate data and the output should be like this:
recordnumber	productname	email
2450	Melona	Jessnie@email.ui


Do you have any solution to our concern using tdenormalize or other component?

Re: How to Delete Duplicates using tDenomalize and not just Merge same value.

MAni5 — Fri, 05 Jul 2024 01:57:44 GMT

Hello Community,
Do you have solutions to my inquiry? I'm having a hard to transform the data.

topic How to Delete Duplicates using tDenomalize and not just Merge same value. in Data Movement & Streaming

How to Delete Duplicates using tDenomalize and not just Merge same value.

Re: How to Delete Duplicates using tDenomalize and not just Merge same value.