topic Re: How to Remove Duplicate Names in QlikView

How to Remove Duplicate Names

krishna20 — Wed, 12 Aug 2015 06:16:00 GMT

Hi Folks,

I'm having Employee_ID which is Identical but Employee_Name getting duplicates from Database itself.I need to restrict it in the Qlikview script .how can we achieve it? Please anyone suggest me. Please find the attached image.

Regards

Krishna

Re: How to Remove Duplicate Names

sunny_talwar — Wed, 12 Aug 2015 06:18:52 GMT

May be try loading the Employee_ID with DISTINCT keyword

Re: How to Remove Duplicate Names

jsingh71 — Wed, 12 Aug 2015 06:25:48 GMT

share some sample data.

Re: How to Remove Duplicate Names

Wed, 12 Aug 2015 06:30:01 GMT

Hi,

Try to find frequency of your EMP_ID (97) if it is giving 2 then load distinct data it will work...

Re: How to Remove Duplicate Names

krishna20 — Wed, 12 Aug 2015 06:42:09 GMT

Hi All,

Thank you for valuable replies. I tried using Distinct keyword in different Scenario's but no use. I have tried at PERSON_ID and EMPLOYEE_ID and EMPLOYEE_NAME.

Employee name is still getting duplicates.If you observe that in the app or above attached image. There is name difference in the both. Please find the attached app.

Regards

Krishna

Re: How to Remove Duplicate Names

Wed, 12 Aug 2015 06:51:19 GMT

Hi,

restrict it with below clause if only single name giving trouble,

FROM

(qvd) where [Employee Full Name]<>'Ray, Ms. Amy (Amy)';

Re: How to Remove Duplicate Names

jonathandienst — Wed, 12 Aug 2015 06:53:53 GMT

You have not explained what you want to do with the duplicates. You could take the first instance of the ID and ignore subsequent duplicates, you could use a date field (if one exists) to choose the most recent, you could ignore the ID and just take the names, you could exception report the duplicates on an integrity report, etc.

So what do you want to do?

Re: How to Remove Duplicate Names

Wed, 12 Aug 2015 07:01:22 GMT

Use DISTINCT keyword or first sorted value

Thanks

Re: How to Remove Duplicate Names

Kushal_Chawda — Wed, 12 Aug 2015 07:01:26 GMT

There is multiple employees name for single Employee ID which is not possible. Check your mapping.

Re: How to Remove Duplicate Names

kavita25 — Wed, 12 Aug 2015 07:05:13 GMT

What is the expected result instead of 8??

Regards,

Kavita

Re: How to Remove Duplicate Names

jonathandienst — Wed, 12 Aug 2015 07:24:52 GMT

It looks like EmployeeID is not the correct key to use for your dataset. Either the key must be a composite of several fields, or you could use the PersonID field, although that is better, it is still not unique. Anyway, I don't think your problem is the technical one of removing duplicates, it is rather the business question of what is the key and how is the data being extracted from the source system.

So you can muck about all you like trying to remove the duplicates, but you are solving the wrong problem.

Re: How to Remove Duplicate Names

krishna20 — Wed, 12 Aug 2015 07:24:56 GMT

Hi Jonathan,

Thank you very much me to think on dates. Exactly there is dates mistake in the database.This Employee has resigned in the past and Joined again .So the Names are repeating but Employee_ID is the Distinct. This is a Instance Database of oracle.

Regards

Krishna

Re: How to Remove Duplicate Names

krishna20 — Wed, 12 Aug 2015 07:29:00 GMT

Hi Jonathan,

I'm not using Employee_ID as a key.I'm using Person _ID as a key.

Re: How to Remove Duplicate Names

jonathandienst — Wed, 12 Aug 2015 07:37:43 GMT

Person_ID is better, but is still not unique. I think you need an extract from your source system that does not produces these duplicate values. If you filter on EmployeeStatus = 'ACTIVE', does that produce unique records for PersonID?

I would examine the SQL query extracting the data. I suspect that there is a bad join in this somewhere. Focus on getting good source data from your source and then you will have less problems with loading and reporting.

Re: How to Remove Duplicate Names

krishna20 — Wed, 12 Aug 2015 07:49:46 GMT

Yes exactly Jonathan, There is a bad join which executed by oracle team in my company. I raised ticket regarding this. They have given a view which is exiting with duplicate data and not a unique Person_ID. Thank you once again Jonathan.