Qlik Sense Data Load - Remove duplicate records

[jerryr125]

Hi - I have the following in my data load editor:

RPSTATUSDATA03:
Noconcatenate
Load *
RESIDENT RPQUISITIONSTATUSDATA02;

I need to remove any records in which there are duplicates.

The field which determines this is the RPNUMMAT field.

If the RPNUMMAT is the same then I do not need the record in RPSTATUSDATA03.

Any thoughts on how I can achieve this process ?

Thanks so much  - Jerry

[rubenmarin · Accepted solution]

Hi, you can use Exists() to check already loaded values:

RPSTATUSDATA03:
Noconcatenate
Load *,
  RPNUMMAT as chkRPNUMMAT 
RESIDENT RPQUISITIONSTATUSDATA02
Where not exists('chkRPNUMMAT ',RPNUMMAT);

DROP Field chkRPNUMMAT;

As all field have been loaded in RPQUISITIONSTATUSDATA02, you have to cahnge field name to check those values loaded in RPSTATUSDATA03.

[jerryr125]

Hi- I finally had the opportunity to try the above - thank you very much - works perfectly !! 

- Jerry

[Dalton_Ruer]

Have you tried LOAD DISTINCT?
Refer to the help documentation here: https://help.qlik.com/en-US/sense/February2021/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptRegu...