Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Posting on Qlik Community is temporarily paused for some users. Thank you for patience and we'll update you soon!
cancel
Showing results for 
Search instead for 
Did you mean: 
its_rajvir
Creator
Creator

Find duplicate rows

Hi Experts,

I need to find the duplicate rows in straight table. For example in below table duplicate rows are with seq no  1,4 for Device 1 and 2,4 for Device 2. Please help.

Seq No. Device Name ID Date Time Txn 
1 Device 1 987654321 15-Mar-23 10:30:20 CI
1 Device 1 987654321 15-Mar-23 10:30:20 CI
2 Device 1 123456789 15-Mar-23 10:34:09 CO
3 Device 1 234512345 15-Mar-23 10:35:46 CO
4 Device 1 654325432 15-Mar-23 10:26:35 CI
4 Device 1 654325432 15-Mar-23 10:26:35 CI
4 Device 1 654325432 15-Mar-23 10:26:35 CI
1 Device 2 473826184 15-Mar-23 10:09:20 CI
2 Device 2 234584632 15-Mar-23 10:18:09 CI
2 Device 2 234584632 15-Mar-23 10:18:09 CI
3 Device 2 837246582 15-Mar-23 10:42:46 CO
4 Device 2 928365101 15-Mar-23 10:55:23 CI
4 Device 2 928365101 15-Mar-23 10:55:23 CI
2 Replies
Gabbar
Specialist
Specialist

Concatenate all fields into 1 field, for this refer to 

https://community.qlik.com/t5/QlikView-App-Dev/How-to-concatenate-all-available-fields-into-single-f...

After that sort the new field in ascending or descending order and then use
if(Field = previous(Field),1,0) as Is_duplicate

MarcoWedel
MVP
MVP

maybe like this?

MarcoWedel_1-1679937271729.png

 

table1:
LOAD Hash128([Seq No.],[Device Name],ID,Date,Time,Txn) as Hash, * 
Inline [
Seq No.	Device Name	ID	Date	Time	Txn 
1	Device 1	987654321	15-Mar-23	10:30:20	CI
1	Device 1	987654321	15-Mar-23	10:30:20	CI
2	Device 1	123456789	15-Mar-23	10:34:09	CO
3	Device 1	234512345	15-Mar-23	10:35:46	CO
4	Device 1	654325432	15-Mar-23	10:26:35	CI
4	Device 1	654325432	15-Mar-23	10:26:35	CI
4	Device 1	654325432	15-Mar-23	10:26:35	CI
1	Device 2	473826184	15-Mar-23	10:09:20	CI
2	Device 2	234584632	15-Mar-23	10:18:09	CI
2	Device 2	234584632	15-Mar-23	10:18:09	CI
3	Device 2	837246582	15-Mar-23	10:42:46	CO
4	Device 2	928365101	15-Mar-23	10:55:23	CI
4	Device 2	928365101	15-Mar-23	10:55:23	CI
] (delimiter is '\t');

Join
LOAD Hash,
     Count(Hash)-1 as DuplicatesNo
Resident table1
Group By Hash;