LOAD just distinct fields from one QVD and store into another qvd

[#1]

Hi Guys,

I have a massive QVD which has around 200 Million records in it. I need to extract just the distinct key from this qvd. The key is just one field called CASE_NO. I've tried the following;

Lets call the original QVD PAYMENT. I first loaded just the keys from this qvd into a table called FULL

FULL:

load

CASE_NO as CASE

FROM

[LOCATION\FULL.qvd]

(qvd);

DISTINCT_KEY:

load

Distinct CASE_NO as KEY

resident FULL;

store DISTINCT_KEY into LOCATION\Distinct.qvd;

The above works but for some reason Distinct.qvd contains ALL keys from the FULL.qvd instead of just the distinct unique ones.

Any help will be appreciated!

Thanks

[#2]

Hi Ali, Please make sure that DISTINCT_KEY table is not concatenated to another Table. I believe you dont need 2 steps. Try Like below:

DISTINCT_KEY:

Noconcatenate

LOAD DISTINCT CASE_NO AS KEY FROM SOURCE.qvd (qvd) ;

STORE DISTINCT_KEY INTO DISTINCT.qvd (qvd) ;

DROP Table DISTINCT_KEY;

(or)

DISTINCT_KEY:

Noconcatenate

LOAD CASE_NO AS KEY FROM SOURCE.qvd (qvd) WHERE NOT EXISTS(CASE_NO);

STORE DISTINCT_KEY INTO DISTINCT.qvd (qvd) ;

DROP Table DISTINCT_KEY;

Please reload the qvw and check the data in qvd.

If this is not working, may be you are doing something wrong and post the actual scipt from qvw.