topic Re: Unexpected charset utf8mb3 for column in Qlik Replicate

Unexpected charset utf8mb3 for column

kritostar — Mon, 01 May 2023 14:23:41 GMT

We created a replication from MySQL to MySQL, and we are getting:

[SOURCE_UNLOAD ]W: Unexpected charset utf8mb3 for column 'collateral_report_final_table.UCCCompanyName' (mysql_endpoint_metadata.c:288)

all the warnings are on the same table, different columns.

Checking both tables on the Bs here are the collations on each:

Source table charset utf8mb3

Destination table charset latin1

And the columns on source are all uyf8mb3 and on destination are utf8

Tried to change all columns collation with a script but still the replication keep changing it to utf8

What can we do to keep this warning from happening?

Re: Unexpected charset utf8mb3 for column

SwathiPulagam — Mon, 01 May 2023 14:31:42 GMT

Hi @kritostar ,

Please add the below Internal parameter to the Target endpoint:

afterConnectScript = SET @@OLD_MODE = REPLACE(@@OLD_MODE, 'UTF8_IS_UTF8MB3', '')

And save the endpoint and reload the tables.

Test this in your lower environment before implementing directly in production.

Thanks,

Swathi

Re: Unexpected charset utf8mb3 for column

yonatanyamin — Thu, 26 Oct 2023 13:01:45 GMT

Hi Swathi,

While the above solution can suppress the warning when using specific versions of Postgres,
There is a solution that allows Replicate to handle the unrecognized character set (UTF8MB3 which was deprecated).

For that, you will need to export the provider syntax of MySQL and add an additional character set to the list -

This is the new charset in the JSON:

{
"charset": "utf8mb4",
"icu_converter": "UTF-8"
},

This is the Old one which the warning is about:

{
"charset": "utf8mb3",
"icu_converter": "UTF-8"
}

I've attached the provider syntax I've used. note that this provider syntax is replacing the default one, so if you already use a modified syntax file, you will need to modify the one you have in place and add the new character set as I did in the attached file.

Thanks,
Yonatan