Data Prep & Duplicates

[Anonymous]

Hi,

Using The Data Prep tool I am trying to find duplicates within a column, is there a way to find those that are an exact match as well as those that are a close match (similar to how excel duplicates and fuzzy match works)?

[aashish_21nov]

tuniqrow will provide list of all duplicate records.

[Anonymous]

Hi Aashish,

Is tuniqrow used in the Talend Data Prep Tool, I don't see can I select this and apply it to a column?

Regards

[aashish_21nov]

yes you can apply it to a column but keep in mind your external link should be duplicate.

[Anonymous · Accepted solution]

Hi,

tUniqueRow is not available in Talend Data Preparation, it solely exists in Talend Studio.

So far in Data Prep, there is no equivalent to tUniqueRow, but addressing deduplication is part of the roadmap. Few things can already help spotting and solving duplicates:

• The charts on the bottom right will show the most common values. Therefore, exact matches (if any) will show up at the top of the chart. See https://help.talend.com/reader/n~7tYndHPhSgSAN0bpenIA/rTPaBMcsHltniefJI2Caaw for more details.
• The function "find and group similar text" performs text clustering with fuzzy matching. See https://help.talend.com/reader/087PIlN0I5BrDxt9016xRA/RVQQzV9CAYXQVh1m6DYWkA

Regards,

Gwendal

[Anonymous]

Thanks Gwendal,
Is there an idea of when this would be in place - I've seen mention of this in some posts in 2016/2017?
Regards,
Darren

[Anonymous]

Hi Darren,

That is something which will be addressed in the course of next year.

Regards,

Gwendal