How To Remove Duplicates and Count Rows?

[pnn44794]

Hello:

I have a dataset that has duplicate entries in the data for LineItemId and LineItemFactId.

As always, any and all help is appreciated.  Thanks in advance.

[Anonymous]

Hi Perry,

Hope the below solution will help you to solve your issue:

Count({<ParentLineItemId-={'-'}>}distinct LineItemId)

Thanks & Regards,

Neelima

[balar025 · Accepted solution]

You can take out those field which are unique combination and then can have sum of flag

like

Load distinct Order_id,

ParentLineItemId,

LineItemId,

1 as lineitemidcount

where not isnull(ParentLineItemId)

use sum(lineitemidcount)

[pnn44794]

Thank you for the reply Neelima.  Sorry for the delay in responding back to you.  Your solution will definitely give me the count but does not remove the duplicates.  As it turns out, the duplicate problem was due to issues with the data itself.  I was verifying that and that's why it took me awhile to respond.

[pnn44794]

Hello Ravi:

Sorry for the delay in replying.  As it turns out, there's a problem with the data itself that's causing the duplicates.  Your solution did remove some of the duplicate lines (where ParentLineItemId is null) and I suspect if not for the data issues, your solution would remove any others.  Thanks for your reply and I'll be marking your answer as the correct answer.  Thanks again.