topic Re: Duplicate Record issue. in QlikView

Duplicate Record issue.

Tue, 21 Feb 2012 02:32:24 GMT

I want to delete duplicate records at time of loading of data only, i.e.,

the script should load only distinct records.

How it is possible.

Moreover on second point even I have data stored in my qvd file, now I can check the duplicate records but also I want to remove them.

Kindly suggest any solution for these two points..

Thanks & Regards.

Nitin Gupta

Duplicate Record issue.

jagan — Tue, 21 Feb 2012 05:28:07 GMT

Hi,

You can use the DISTINCT option while loading the data

TableName:

LOAD DISTINCT

Field1,

Field2,

FROM DataSource;

Second, Load the data in the qvd temporarily to a table and avoid duplicates in the table by using the DISTINCT keyword during load and now Save the data to the existing QVD by using the STORE command.

Hope this helps you.

Regards,

Jagan.

Duplicate Record issue.

Tue, 21 Feb 2012 07:14:37 GMT

Thanks Jagan ,

But I think Distinct treats all columns in one go and checks the duplicacy

But I want to define some specific columns on behalf of which duplicacy should be checked and unwanted records should be removed.

I think now you may get my point.

Please suggest your perception if I am not wrong, is it possible?

Duplicate Record issue.

Tue, 21 Feb 2012 07:37:42 GMT

Hi,

I am curious to see situation where you have such requirement!

QV creates relationships between whatever fields that are supplied to it as database.

If randomly selected columns are filtered, whole database is supposed to get filtered.

It is same like applying "Filter" in Excel.

Filter gets applied to all columns in given table.

Do u agree?

Duplicate Record issue.

jagan — Tue, 21 Feb 2012 07:42:24 GMT

Hi,

How you filter records with Distinct values by paritcular column in a table. Explain your requirement with an example, it would be helpful to find solution for this.

Regards,

Jagan.

Re: Duplicate Record issue.

shantanu73 — Tue, 12 Jun 2012 09:47:56 GMT

Hello Jagan Mohan,

I had gone through the above trail mail. The advice/suggession/idea which you had given for dumping the distinct record in QVD file is good.

But mistakely I had not used Parking table in QV_Table file for putting the distinct record and directly I had store the data from database table to QVD file. And Now, I want to remove all record or duplicate record from QVD file.

Can you advice/suggest/idea to me that how I should remove all record or duplicate record from QVD file?.

Thanks in Advance

Shantanu

Re: Duplicate Record issue.

shantanu73 — Tue, 12 Jun 2012 09:48:07 GMT

Hello Jagan Mohan,

I had gone through the above trail mail. The advice/suggession/idea which you had given for dumping the distinct record in QVD file is good.

Can you advice/suggest/idea to me that how I should remove all record or duplicate record from QVD file?.

Thanks in Advance

Shantanu

Re: Duplicate Record issue.

jagan — Tue, 12 Jun 2012 10:15:58 GMT

Hi,

First load data in QVD to Qlikview table.

TableName:

Load DISTINCT

FROM QVDFile;

Now replace the QVD file with the table created above which has Distinct records.

STORE TableName INTO QVDFile;

Hope this helps you.

Regards,

Jagan.

Re: Duplicate Record issue.

Tue, 12 Jun 2012 12:52:19 GMT

If my understanding is correct u r looking for the distinct values of field.

then u can use code like

Tab:

LOAD A,

FROM

C:\Examples\Distinct.xlsx

(ooxml, embedded labels, table is Sheet1)

where not exists(A);

Re: Duplicate Record issue.

shantanu73 — Wed, 13 Jun 2012 04:38:52 GMT

Hello Jagani,

The process is very good when record is less. But when there is Cores or Billions of records in that case it will be time consuming process.

Is it possible to delete unwanted record directly from QVD files?. The way we are doing on Database Table.

Regards

Shantanu

Re: Duplicate Record issue.

jagan — Wed, 13 Jun 2012 09:20:35 GMT

Hi,

As per my knowledge, I think it is not possible to directly remove some rows from QVD without loading and saving into it as we do in database.

Regards,

jagan.

Re: Duplicate Record issue.

shantanu73 — Wed, 13 Jun 2012 10:13:22 GMT

Hello Jagan,

Lot of Thanks for quick respons

Thanks & Regards

Shantanu