data duplication

[Anonymous]

fielda	fieldb	fieldc	fieldD
2	abc	2	123
2	abc	2	456
2	dfg	3	123
2	dfg
2	hij	4	123
2	hij	4	456
2	klm	5	123
2	klm	5	456
2	nop	7	123
2	nop	7	456

output shud be :

2	abc	2	123
2	hij	4	123
2	klm	5	123
2	nop	7	123

[sunny_talwar]

May be load with DISTINCT keyword

LOAD Distinct

     fielda,

     fieldb,

     fieldc

FROM ....

[Anonymous]

taking only distinct will not work.

[marcus_sommer]

What is the condition for "fieldD"?

- Marcus

[datagrrl]

I have some questions about your model. Is this a single flat table? Is the duplication in the data or being caused by the association?

Do you want the max value of fieldc?

[Anonymous]

it is just the single table now,

[Anonymous]

kindly help with the resolution

[Anonymous]

duplicacy should be removed from field b

[Anonymous]

i want sum of fieldc

[Kushal_Chawda]

try this

Data:

LOAD Distinct

     fielda,

     fieldb,

     fieldc,

     fieldd

FROM ...

Final

LOAD

     fielda,

     fieldb,

     fieldc,

     min(fieldd) as fieldd

resident Data

group by fielda,fieldb,fieldc;

drop table Data;