topic Re: tmomoutput: Issues with special characters in Talend Studio

tmomoutput: Issues with special characters

Anonymous — Thu, 12 Sep 2019 18:53:40 GMT

I am facing issues while publishing special characters in UTF-8 format into the mqueue(WebSphere MQ) using tmomoutput component, Below is the error.

Also I have share few screen shot of the configured tmomoutput component. Can someone please help on how to resolve this?

java.nio.charset.UnmappableCharacterException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:282)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:816)
at com.ibm.mq.jmqi.system.JmqiCodepage.stringToBytes(JmqiCodepage.java:703)
at com.ibm.mq.MQMessage.writeString(MQMessage.java:2746)

Re: tmomoutput: Issues with special characters

fdenis — Fri, 13 Sep 2019 08:22:15 GMT

what is your mq configuration, can you manually add utf8 char?

Re: tmomoutput: Issues with special characters

Ineedtalend — Tue, 08 Dec 2020 23:02:41 GMT

fdenis,

I run into the same issue while using tMomOutput. Could you please shed some light if you were able to resolve this?

Thanks.

Re: tmomoutput: Issues with special characters

SnowFrog — Wed, 08 Jun 2022 15:57:39 GMT

Has anybody ever answered this? I have the same error for the same component and yes it is a UTF-8 file but only when they get big. Small files below a meg are processed ok. Big in the 3MB range fail with that error?

I do not know what is the character set of a file before I receive it. Can be ascii, 8859 or UTF-8.

Re: tmomoutput: Issues with special characters

SnowFrog — Mon, 08 May 2023 21:34:07 GMT

This is still causing me issue. I'm using and MQ Server. I'm specifying in the advance option the use of the characterSet to be 1208 which is IBM's CCSID for UTF-8. But it is still giving me errors. It will take all the normal french characters with nothin else, otherwise I get the same error as the subject of the original question.

What am I doing wrong?

Over 400 people looked at this thread and no one has a clue.

Re: tmomoutput: Issues with special characters

SnowFrog — Tue, 09 May 2023 22:43:01 GMT

I will answer my own questions to avoid other the torture.

The issue is that although the MQ server defaults to UTF-8 the tMOMOutput defaults to the charset of he local machine on which it runs.

To ensure that the message you are sending is converted to UTF-8 you have to do both

set the "Use Format" as in the picture in the initial post

and as well you have to select "Set MQMD Fields"

If you are lucky the panel that expends down from there will already show you the Left field set to "characterSet" with a value of 1208 in the right field.

This is actually correct 1208 is the IBM CCSID for UTF-8 (of course they had to rename something already standard :-()

If you don't see this in the panel that opens select the left fields and you will see a bunch of MQMD fields that you can select. Select characterSet and give it a value of 1208.