topic Re: tADDCRCRow component duplicate CRC issue in Talend Studio

tADDCRCRow component duplicate CRC issue

TalendUser21 — Fri, 15 Nov 2024 22:47:11 GMT

Hi,

I am currently trying to load about 3 million records into my target table and I used tADDCRCRow component to generate a HASH key with CRC type set to CRC32

The tADDCRCRow component is configured to generate CRC values based on 2 input fields : PRODUCT_ID and DATE.

But I noticed that there are several duplicate CRC value has been generated for different set of Key combination.

Do you have any suggestion or idea why this is behaving this way?

Re: tADDCRCRow component duplicate CRC issue

InfoCraft — Mon, 27 Jun 2022 13:47:36 GMT

Hi @Not defined Not defined,

are you sure you are using CR32? because CRC=309142 is 10^5(for 3 million rows it's normal to have duplicate CRC).

To avoid duplicates, your CRC must be at least 10^6

Re: tADDCRCRow component duplicate CRC issue

TalendUser21 — Mon, 27 Jun 2022 19:01:36 GMT

Yes, I am using CRC32 and I do get CRC as long 2616349547 and as short as 309142.