tADDCRCRow component duplicate CRC issue

[TalendUser21]

Hi,

I am currently trying to load about 3 million records into my target table and I used tADDCRCRow component to generate a HASH key with CRC type set to CRC32

The tADDCRCRow component is configured to generate CRC values based on 2 input fields : PRODUCT_ID and DATE.

But I noticed that there are several duplicate CRC value has been generated for different set of Key combination.

Do you have any suggestion or idea why this is behaving this way?

[InfoCraft · Accepted solution]

Hi @Not defined Not defined​,

are you sure you are using CR32? because CRC=309142 is 10^5(for 3 million rows it's normal to have duplicate CRC).

To avoid duplicates, your CRC must be at least 10^6

[TalendUser21]

Yes, I am using CRC32 and I do get CRC as long 2616349547 and as short as 309142.