topic Re: Code to clean up duplicates in QlikView

Code to clean up duplicates

hobanwashburne — Tue, 02 Sep 2014 14:32:11 GMT

I have quite a few joins in my qvd creation files and have been having trouble with duplicated records. Shouldn't the following code stuck at the end of my creation (just before the store command) help with that? If so, wouldn't it be a good idea to use it in most of my qvd creation files? Does anyone else use something like this?

MasterFile:

LOAD Distinct *

Resident Temp;

DROP Table Temp;

STORE MasterFile into ....

Re: Code to clean up duplicates

MK_QSL — Tue, 02 Sep 2014 14:41:34 GMT

Temp:

Load * From.....

NoConcatenate

MasterFile:

LOAD Distinct *

Resident Temp;

DROP Table Temp;

STORE MasterFile into ....

Re: Code to clean up duplicates

marcus_sommer — Tue, 02 Sep 2014 14:45:22 GMT

With distinct you eliminated duplicated rows but you could still have duplicates in key-fields (same id but several matchings). For things like this or to have a possibilty to work on the reasons you need to identify these rows, maybe with something like this:

duplicate:

Load ID, if(count(ID) > 1, 'duplicate', 'ok') as check From x;

- Marcus