how to find duplicates

[#1]

Hi Experts,

i have a document which has Passenger ID, Journey ID, Direction,  respective calendar for year, month.

Each Passenger will have unique passenger ID & each journey will have unique Journey ID and direction will be either inward or outward.

One passenger may have multiple journeys on the same day but they will be either inward or outward.

A passenger cannot have multiple journeys on a same day for inward alone (or outward alone). this is called duplicate entry.

i need to report the duplicate entries in a table(chart).

how can i bring duplicate passenger ids alone in a chart.

Please help me to find solution.

[Anonymous]

Hi

How about creating a unique key made up of the passenger id, journey id, direction and date field and then do resident load adding the count of this key to your data table and then use that in your chart?

[jonathandienst]

Do you want to find multiple inwards without corresponding outwards (and vice versa)? Or do you just want to find multiples regardless of the other direction?

[#4]

hi thanks for the reply.

i cant change the data model.

i need to do this in chart only. is it possible?

[Anonymous]

Another option would be to create a unique key as described previously and then use the previous function to create a flag where the unique key matches the flag in the previous record loaded.

[#6]

i want to find multiple journeys for inward, multiple journeys for outward in a same day

[santiago_respane · Accepted solution]

Hi,

you can use something like this, a chart with Passenger ID, Journey ID, JourneyDayField, Direction as dimensions and something like this as expression:

=SUM(Aggr(if(count(Journey ID)>1,1,0),Passenger ID,Journey ID,JourneyDayField,Direction))

Please let me know if this helps.

Kind regards,

[jonathandienst]

I would prefer to do some work in the back end, as the performance in the front end might suffer if the data set is large. You could try this:

     Dimension: Passenger ID, Date

     Expression:

     If(Fabs(Count({<Direction = {'outward'}>} [Journey ID]) - Count({<Direction = {'inward'}>} [Journey ID])) > 1, 'Duplicates', null())

[effinty2112]

Hi Santiago,

               Shouldn't we take out JourneyID in the list of AGGR dimensions?

=SUM(Aggr(if(count(Journey ID)>1,1,0),Passenger ID,JourneyDayField,Direction))

BTW I think your idea is a really good one.

Cheers

Andrew

[santiago_respane]

Hi Andrew,

I've tested and it doesn't change the final result in this case, but indeed it makes no sense that Journey ID is in the AGGR dimensions.

nice observation and thanks for the feedback!

Kind regards,