I have an app with multiple fact tables which i am trying to optimize. For that purpose trying to create a link table (have not included that part)
For doing this i am doing a load distnct
load
Distinct
Employee_ID,
Parent,
FactType
resident
MasterData
where FactType='xyz';
Master table has approx 120 million records.
the result of a expression count(distinct Employee_ID&Parent) is around 400k records
But somehow the load distinct doesnt work. the above load results in 120 million records
its weird. i remember reading somewhere about this but couldnt find the article again
but basically i replace above script with
load
Distinct
Employee_ID,
Parent,
MinString(FactType)
resident
MasterData
where FactType='xyz'
group by Employee_ID , Parent;
this one worked perfectly.
Can someone explain theory behind this?
@rwunderlich @Oleg_Troyansky
