Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
khaycock
Creator
Creator

Remove Duplicates in Load

I have two fields that I need to make a key on and remove any duplicates of this key. How can I do this? 

E.g:

Table1:

LOAD

Plugin,
[IP Address],
Plugin&[IP Address] AS KEY,
Field1,
Field2

FROM DATA

Labels (1)
4 Replies
Shubham_Deshmukh
Specialist
Specialist

WHat do you mean by remove duplicate?
khaycock
Creator
Creator
Author

There are duplicated versions of the KEY within the data. I want to remove that line of data.

khaycock
Creator
Creator
Author

Tenable:
LOAD Plugin,
[IP Address],
[IP Address] & '_' & Plugin AS KEY1 ,
Replace([Patch Publication Date],'UTC','') as [Patch Publication Date],
(Replace([Last Observed],'UTC','')) as [Last Observed],
(Replace([First Discovered],'UTC','')) as [First Discovered]

FROM
[data.csv]
(txt, codepage is 1252, embedded labels, delimiter is ',', msq);


Tenable_temp:
load distinct

*,
date(today(1)) - date(floor(date#([First Discovered],'MMM DD, YYYY hh:mm:ss')),'DD/MM/YYYY') as VulAgeNo,
if(date(today())-7 <= date(floor(date#([Last Observed],'MMM DD, YYYY hh:mm:ss'))), 'Scanned Recently', 'Re-Scan Needed') as ReScan
RESIDENT Tenable
WHERE NOT EXISTS (KEY1,[IP Address] & '_' & Plugin);

Drop table Tenable;

I think I followed the steps properly but this returns no results. Is it because I am forcing the key in the script?