Skip to main content
Announcements
See what Drew Clarke has to say about the Qlik Talend Cloud launch! READ THE BLOG
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Eliminating duplicate records: Load distinct or using alternative methods?

Hi all,

I am facing the following situation.

I want to make a reload of a dashboard. The problem is I found out there are several duplicate records/transactions for one specific country. Now I want to eliminate those duplicate records by making use of the "Load Distinct" function.

However, this does not show the desired results in the sense that the data is no longer valid with some reference points I got in the data.

So I was wondering, what are the pros and cons of this method? Are there any alternatives regarding this situation?

I was also thinking about making use of the "Group by" function or to let the table join with itself.

How to tackle this problem? Please share your opinions

Regards,

1 Reply
puttemans
Specialist
Specialist

Hi,

I think the load distinct is one of the better ways forward, but you need to make sure that you are really only eliminating similar records. Therefore, you may need to first concatenate some variables to create a key you can then compare with the 'distinct' function.

Group by will just group/count all similar records based on the value you select, but will not exclude them.