topic [resolved] Encoding problems during conversion of XML to MySQL in Talend Studio

[resolved] Encoding problems during conversion of XML to MySQL

Anonymous — Sat, 16 Nov 2024 11:34:44 GMT

Hi guys,
I've tried hours to solve this, but I am already going nuts here. I have a public XML file () that I want to fetch and map to a MySQL table (columns: id, name, address, country, list). So I use this:
tFile_Fetch --onSubJobOk--> tFileInputXML --> tMap --> MySQLOutput
It works fine. All rows are written to MySQL, but cyrillic characters etc. are shown as multiple question marks in MySQL. So I thought it has to be an encoding problem (like the XML being non UTF-8) but the XML says "encoding=UTF-8", all options in Talend are set to UTF-8 and the MySQL database and the tables use charset "utf8" with collation "utf8_unicode_ci". When I open the XML file in a simple text editor, I also see the correct chars. I just doesn't work when writing them to MySQL.
Do you have an idea why this could happen? This is driving me crazy. I would really appreciate your help
btw.: using Talend Open Studio 5.5.1
thx

Re: [resolved] Encoding problems during conversion of XML to MySQL

Anonymous — Sun, 20 Jul 2014 18:48:49 GMT

The encoding is only a matter of parsing the XML file. The job internal converts all char sets to UTF-16 (Java default encoding).
The database driver takes that UTF-16 String an sends it to the database.
If you have configured your table as UTF-8 everything should be fine.
I suggest for your tests you write cyrillic letters with another tool in your database (e.g. use the MySQL clients) and check if you could read it correctly.
You should - for tests - change the output from the tMysqlOutput to a tFileOutputDelimited (also configured as UTF-8) and check if you can read this file correctly.
I guess the read xml is not correctly configured.

Re: [resolved] Encoding problems during conversion of XML to MySQL

Anonymous — Tue, 29 Jul 2014 17:29:51 GMT

Thanks jlolling, I've found the error: If anybody is experiencing the same problems, try adding the following parameters to your JDBC Connection (in my case the MySQL Connection parameters): characterEncoding=utf8&characterSetResult=utf8&useUnicode=true. This worked for me.
Thanks again

Re: [resolved] Encoding problems during conversion of XML to MySQL

_AnonymousUser — Mon, 06 Apr 2015 11:25:51 GMT

Hello All,
I am using Talend 5.5.1 and facing problem with few characters like Å etc. I have done the following steps but still facing the issue
- Changed Talend encoding type to utf8
- Database connection I changed it to - characterEncoding=utf8&characterSetResult=utf8&useUnicode=true
but unable to resolve the error.
Any help would be greatly appreciated