UTF-8 Error while bulk loading data with japanese chars to AWS MySQL Database

[fabiosurini]

Hi

we have a MySQL 5.7 on AWS with utf8mb4 tables.

We use the DBBulk task to upload data in these tables but we get

java.sql.SQLException: Invalid utf8 character string: '"<some japanese ch>

Note that:

• db, tables and column are all set with

  utf8mb4 char set

• in the jdbc connection string we add "useUnicode" and "charset=utf8" params

What can be the cause of this issue?

Thanks

Fabio

[Anonymous]

Hello @Fabio Surini​ ,

Could you please try to run the below query in the AWS db to check if all the settings are utf8?

if some are not utf8, then try to change them to utf8 and apply to the db?

show variables like "character%";

[Anonymous]

Hi

Try to add these parameters in the additional JDBC parameter field.

useUnicode=true&characterEncoding=utf8

If the data are read from file, select encoding 'UTF-8' on the tFileInputXXX component.

Regards

Shong

[fabiosurini]

Hi

we checked all parameters and all is set as utf8 / utf8mb4

The issue seems to be related to a chinese character included in the "CJK Unified Ideographs Extension B" and not mapped in MySQL charset...

Fabio