Incremental load

[GOKULAKANNAN]

I have to incremental load for below scenario,

It should insert and update

Id, place,count

1,chennai,10

1,chennai,11

1,chennai,10

2,madurai,10

2,madurai,11

2,madurai,10

In this i need to do increment load, where it should not remove duplicates but if any update is there it should do. Primary key can be created by concatenating all 3 columns, but i need all 6 records to be presented.

[deepanshuSh]

Hi, since you have common data in all three fields with repetitions what you can do is use rowNo () function to create an addition column and then concatenate them together to create a composite key through which you can do the required incremental load. for the update part you need to direct it towards the Id as key if possible.

[rwunderlich]

You'll need to create a new field to serve as the primary key. 

Hash128(Id, place, count) as Key

-Rob