Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
its_rajvir
Creator
Creator

Find duplicate rows

Hi Experts,

I need to find the duplicate rows in straight table. For example in below table duplicate rows are with seq no  1,4 for Device 1 and 2,4 for Device 2. Please help.

Seq No. Device Name ID Date Time Txn 
1 Device 1 987654321 15-Mar-23 10:30:20 CI
1 Device 1 987654321 15-Mar-23 10:30:20 CI
2 Device 1 123456789 15-Mar-23 10:34:09 CO
3 Device 1 234512345 15-Mar-23 10:35:46 CO
4 Device 1 654325432 15-Mar-23 10:26:35 CI
4 Device 1 654325432 15-Mar-23 10:26:35 CI
4 Device 1 654325432 15-Mar-23 10:26:35 CI
1 Device 2 473826184 15-Mar-23 10:09:20 CI
2 Device 2 234584632 15-Mar-23 10:18:09 CI
2 Device 2 234584632 15-Mar-23 10:18:09 CI
3 Device 2 837246582 15-Mar-23 10:42:46 CO
4 Device 2 928365101 15-Mar-23 10:55:23 CI
4 Device 2 928365101 15-Mar-23 10:55:23 CI
2 Replies
Gabbar
Specialist
Specialist

Concatenate all fields into 1 field, for this refer to 

https://community.qlik.com/t5/QlikView-App-Dev/How-to-concatenate-all-available-fields-into-single-f...

After that sort the new field in ascending or descending order and then use
if(Field = previous(Field),1,0) as Is_duplicate

MarcoWedel

maybe like this?

MarcoWedel_1-1679937271729.png

 

table1:
LOAD Hash128([Seq No.],[Device Name],ID,Date,Time,Txn) as Hash, * 
Inline [
Seq No.	Device Name	ID	Date	Time	Txn 
1	Device 1	987654321	15-Mar-23	10:30:20	CI
1	Device 1	987654321	15-Mar-23	10:30:20	CI
2	Device 1	123456789	15-Mar-23	10:34:09	CO
3	Device 1	234512345	15-Mar-23	10:35:46	CO
4	Device 1	654325432	15-Mar-23	10:26:35	CI
4	Device 1	654325432	15-Mar-23	10:26:35	CI
4	Device 1	654325432	15-Mar-23	10:26:35	CI
1	Device 2	473826184	15-Mar-23	10:09:20	CI
2	Device 2	234584632	15-Mar-23	10:18:09	CI
2	Device 2	234584632	15-Mar-23	10:18:09	CI
3	Device 2	837246582	15-Mar-23	10:42:46	CO
4	Device 2	928365101	15-Mar-23	10:55:23	CI
4	Device 2	928365101	15-Mar-23	10:55:23	CI
] (delimiter is '\t');

Join
LOAD Hash,
     Count(Hash)-1 as DuplicatesNo
Resident table1
Group By Hash;