Data Quality Checking

[#1]

Hi all,

when you are working with a database with millions of records,  how do you check that your dashboards do not contain errors?, ie. some records have been lost, some undesired outer join duplicates info, ...

In fact I find that it is easier when a BIG mistake has been made because it's easy to identify it than when figures seem ok but in reality they are not...

Tips and advices are more than welcomed about this critical point!

Thanks

[alexandros17]

Good question,

data must be ALWAYS checked by final user that know is Sales or invoices for example are correct;

However you can verify data by doing a specific atomic selection and compare results with what final user want.

This is my way of testing documents

hope it helps

[#3]

Good question of the day !

I think it depends on your architecture...

if you have 3 tiers, and you are forming STAR schema.

then in the first tier, you can load the data, and at the same time create the key for the schema here. and a simple check between the total rows with the key will tell u if your data is good.

Just another possible way ...