Problem with Duplicate records

[#1]

Hi Buddy,

I have a table with 2918117 records. I converted database table in qvd when i'm check the raw data on a particular date the records are 6399.After i made some transformations i'll check the same date the record shows 90000+.Guys please help me why I'm getting like this.

[Colin-Albert]

It would help if you posted your script or the changes you made that caused the record count to increase.

[sunny_talwar]

It may be been caused to Cartesian product of you are doing a join before storing it into a qvd. I would investigate any join before storing it into a qvd.

[sunny_talwar]

And probably use Mapping Loads wherever it is possible to use them.

[#5]

Noted and Thanks Colin but present i don't have access the server once i get access definitely post the screenshots

[stevedark]

I would agree with sunindia‌ that it is likely to be a join that is causing the increase.

As a general rule of thumb you do not need to make joins at all in your load script.  You can simply keep the tables separate and allow associations to happen in the data model.

Using mapping loads are a good way of avoiding joins for simple lookup tables (but if duplicates are being caused mapping tables would not work as only one value would be mapped).

This article on mapping may be of interest to you:

http://www.quickintelligence.co.uk/applymap-is-it-so-wrong/

Steve

[Colin-Albert]

Further to Steve's post, this article on mapping tables is useful.

Don't join - use Applymap instead

[#8]

Thanks guys for your support