Duplicate Record issue.

[#1]

I want to delete duplicate records at time of loading of data only, i.e.,

the script should load only distinct records.

How it is possible.

Moreover on second point even I have data stored in my qvd file, now I can check the duplicate records but also I want to remove them.

Kindly suggest any solution for these two points..

Thanks & Regards.

Nitin Gupta

[jagan]

Hi,

You can use the DISTINCT option while loading the data

TableName:

LOAD DISTINCT

     Field1,

     Field2,

     '

     '

     '

FROM DataSource;

Second, Load the data in the qvd temporarily to a table and avoid duplicates in the table by using the DISTINCT keyword during load and now Save the data to the existing QVD by using the STORE command.

Hope this helps you.

Regards,

Jagan.

[#3]

Thanks Jagan ,

But I think Distinct treats all columns in one go and checks the duplicacy

But I want to define some specific columns on behalf of which duplicacy should be checked and unwanted records should be removed.

I think now you may get my point.

Please suggest your perception if I am not wrong, is it possible?

[#4]

Hi,

I am curious to see situation where you have such requirement!

QV creates relationships between whatever fields that are supplied to it as database.

If randomly selected columns are filtered, whole database is supposed to get filtered.

It is same like applying "Filter" in Excel.

Filter gets applied to all columns in given table.

Do u agree?

[jagan]

Hi,

How you filter records with Distinct values by paritcular column in a table.  Explain your requirement with an example, it would be helpful to find solution for this.

Regards,

Jagan.

[shantanu73]

Hello Jagan Mohan,

I had gone through the above trail mail. The advice/suggession/idea which you had given for dumping the distinct record in QVD file is good.

But mistakely I had not used Parking table in QV_Table file for putting the distinct record and directly I had store the data from database table to QVD file. And Now, I want to remove all record or duplicate record from QVD file.

Can you advice/suggest/idea to me that how I should remove all record or duplicate record from QVD file?.

Thanks in Advance

Shantanu

[shantanu73]

Hello Jagan Mohan,

I had gone through the above trail mail. The advice/suggession/idea which you had given for dumping the distinct record in QVD file is good.

But mistakely I had not used Parking table in QV_Table file for putting the distinct record and directly I had store the data from database table to QVD file. And Now, I want to remove all record or duplicate record from QVD file.

Can you advice/suggest/idea to me that how I should remove all record or duplicate record from QVD file?.

Thanks in Advance

Shantanu

[jagan]

Hi,

First load data in QVD to Qlikview table.

TableName:

Load DISTINCT

     *

FROM QVDFile;

Now replace the QVD file with the table created above which has Distinct records.

STORE TableName INTO QVDFile;

Hope this helps you.

Regards,

Jagan.

[#9]

If my understanding is correct u r looking for the distinct values of field.

then u can use code like

Tab:

LOAD A,

     B,

     C,

     X

FROM

C:\Examples\Distinct.xlsx

(ooxml, embedded labels, table is Sheet1)

where not exists(A);

[shantanu73]

Hello Jagani,

The process is very good when record is less. But when there is Cores or Billions of records in that case it will be time consuming process.

Is it possible to delete unwanted record directly from QVD files?. The way we are doing on Database Table.

Regards

Shantanu