topic Re: UTF-8 Error while bulk loading data with japanese chars to AWS MySQL Database in Talend Studio

UTF-8 Error while bulk loading data with japanese chars to AWS MySQL Database

fabiosurini — Fri, 15 Nov 2024 22:17:35 GMT

we have a MySQL 5.7 on AWS with utf8mb4 tables.

We use the DBBulk task to upload data in these tables but we get

java.sql.SQLException: Invalid utf8 character string: '"<some japanese ch>

Note that:

What can be the cause of this issue?

Thanks

Fabio

Anonymous — Thu, 08 Dec 2022 02:35:56 GMT

Hello @Fabio Surini ,

Could you please try to run the below query in the AWS db to check if all the settings are utf8?

if some are not utf8, then try to change them to utf8 and apply to the db?

show variables like "character%";

Anonymous — Thu, 08 Dec 2022 07:05:48 GMT

Try to add these parameters in the additional JDBC parameter field.

useUnicode=true&characterEncoding=utf8

If the data are read from file, select encoding 'UTF-8' on the tFileInputXXX component.

Regards

Shong

fabiosurini — Mon, 12 Dec 2022 16:06:18 GMT

we checked all parameters and all is set as utf8 / utf8mb4

The issue seems to be related to a chinese character included in the "CJK Unified Ideographs Extension B" and not mapped in MySQL charset...

Fabio