topic Re: tOracleInput - encoding parameter not available anymore in Talend Studio

tOracleInput - encoding parameter not available anymore

bi-dev-1 — Fri, 15 Nov 2024 23:18:49 GMT

Hello,

I migrated to Talend Open Studio for Data Integration version 7.3.1 (from version 6.5.1)

The parameter "Encoding" of the tOracleInput component is now masked as we can see in the file : tOracleInput_java.xml

<ADVANCED_PARAMETERS>

<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="100"

REQUIRED="true" SHOW_IF="USE_EXISTING_CONNECTION == 'false'"

SHOW="false"

</PARAMETER>

</ADVANCED_PARAMETERS>

How can I set the encoding type now that the parameter is not available anymore ?

I get

ISO-8859-15 strings from my result set while I want UTF-8 strings.

Due to that fact, the data comparison is tricky and requires

this kind of formula :

"ôéà".equals(new String(row1.column1.getBytes("ISO-8859-15"), "UTF-8") )

I would prefer a global solution rather than the usage of such formulas with which I'm not comfortable.

Regards

Re: tOracleInput - encoding parameter not available anymore

Anonymous — Fri, 14 Jan 2022 09:10:25 GMT

@samuel verheecke , I don't know why this option is hidden on version7.3.1, if you want to make this option is visible, modify the parameter SHOW="false" to SHOW="true" in tOracleInput_java.xml file, and restart studio.

Regards

Shong

Re: tOracleInput - encoding parameter not available anymore

bi-dev-1 — Fri, 14 Jan 2022 10:29:44 GMT

Thanks for your response.

The parameter seems to be masked because it has no effect now.

I unmasked the parameter and I selected UTF-8 in the component configuration.

The returned value is still encoded in ISO-8859-15.

This feature may be implemented in a different way on version 7.3.1.

Please let me know.

Re: tOracleInput - encoding parameter not available anymore

bi-dev-1 — Fri, 14 Jan 2022 14:34:40 GMT

Sorry, I made a mistake identifying the problem.

The returned value is well encoded in UTF-8.

It is auto-detected by the driver (parameter not needed).

My issue seems to be related to the encoding of my job.

row1.column1 = value "ôéà" returned from the database

new String(row1.column1.getBytes("UTF-8"), "UTF-8") returns "ôéà"

new String("ôéà".getBytes("UTF-8"), "UTF-8") returns "???"

I don't understand where the problem comes from.

Preferences > General > Workspace > Text file encoding = UTF-8

TOS.ini contains : -Dfile.encoding=UTF-8

My JAVA platform is OpenJDK 11

Re: tOracleInput - encoding parameter not available anymore

Anonymous — Mon, 17 Jan 2022 02:28:02 GMT

@samuel verheecke , I don't know why you need this formula in your job.

new String("ôéà".getBytes("UTF-8"), "UTF-8") returns "???"

If you need to change the job execution encoding, try to change it from the job level.

Click the Run tab > Advanced settings > select Use specific JVM arguments > New > enter -Dfile.encoding=UTF-8. Then regenerate and redeploy the task from TAC.

Regards

Shong

Re: tOracleInput - encoding parameter not available anymore

bi-dev-1 — Mon, 17 Jan 2022 09:11:08 GMT

Thanks for your help.

The link that you provided does not work :

"Oops! Looks like we ran into a problem with your request. Please contact Talend Customer Support for further assistance."

Originally, I need to compare "ôéà" and row1.column1

I know row1.column1 is equal to ôéà

But "ôéà".equals(row1.column1) fails.

new String(row1.column1.getBytes("UTF-8"), "UTF-8") was just to check the encoding of the value returned by the database.

The issue comes from the value I typed in the job.

new String("ôéà".getBytes("UTF-8"), "UTF-8") should return : ôéà (instead of ???)

Re: tOracleInput - encoding parameter not available anymore

Anonymous — Tue, 18 Jan 2022 09:36:54 GMT

@samuel verheecke , sorry, I didn't notice it is an internal article, I have updated my post to include the article content, can you try to update the encoding from job level and test again?