Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Not applicable

Remove duplicate fieldvalue

Searched various posts regarding this issue but couldn't find any solution yet.

Due to an error in a 3rd party datadump I have various duplications in a sold volume field but they all have a different unique selling ID.

It is hard to repair the bug from 3rd party so I hope it can by solved by a loading script.

I like to keep one value of course and delete all others till next unique value appaers.

I attached the input source.

Looking forward for any reaction.


Bert (rooky user)

3 Replies

Re: Remove duplicate fieldvalue

You could with peek() or previous() check values from former rows against the values from the current row and then set a flag which you could use to filter your data (per selection in gui or per where-clause within the script) - how it works: Pick() or Previous() ?

- Marcus

Honored Contributor III

Re: Remove duplicate fieldvalue

use distinct

load distinct


resident table;



Re: Remove duplicate fieldvalue



load *

From Table;



load *,

if(rowno()=1,1, if( previous(Field1)=Field1 and previous(Field2)=Field2 and previous(Field3)=Field3,0,1)) as Flag

resident Data

order by ID; // Here you have to do order by with key

Drop table Data;

Here, I have taken random 3 fields but in your case you will have all the Fields of that table. So you will have if condition for all the fields of the table



load *

resident Data

where Flag =1;

drop table New;

Community Browser