topic (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error) in Talend Studio

(SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Wed, 26 Jul 2017 10:28:12 GMT

Hi, i have this problem with tMysqlOutput.

tMysqlOutput_3 Data truncation: Data too long for column 'APPLICATION' at row 85

I have set these additional JDBC parameters:

noDatetimeStringSync=true&useUnicode=true&characterEncoding=utf8

and the problem is resolved in the Studio but on server continues to persist.

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Wed, 26 Jul 2017 13:28:08 GMT

Update: I have try to execute the job with Studio on the Server virtual machine and run succesfully.

But the problem when executing in the Talend Administration Console persist..

Error:

[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection attempts to '172.16.72.245' with the user : 'AOSIS'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection to '172.16.72.245' with the user : 'AOSIS' has succeeded.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Attempt to get bapi function : 'ZDW_MATERIAL'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Success to get bapi function : 'ZDW_MATERIAL'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Execute the bapi function : 'ZDW_MATERIAL' has finished.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Closing the connection.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection has closed.
[FATAL]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tMysqlOutput_3 Data truncation: Data too long for column 'APPLICATION' at row 85
[statistics] disconnected
Exception in component tMysqlOutput_3
java.sql.BatchUpdateException: Data truncation: Data too long for column 'APPLICATION' at row 85
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.Util.getInstance(Util.java:387)
	at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1154)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1582)
	at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1248)
	at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.tFixedFlowInput_1Process(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:7676)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.runJobInTOS(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:9426)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.main(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:9251)

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 04:12:41 GMT

Hi,

What's the locale encoding of your TAC server? If the project starts with the -Dfile.encoding=UTF-8 set, try to change the locale encoding to UTF-8.

Let us know if it is OK with you.

Best regards

Sabrina

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 08:30:55 GMT

Hi, the studio configuration file is:

-vm
C:\Program Files\Java\jdk1.8.0_121\bin\javaw.exe
-vmargs
-Xms1024m
-Xmx2048m
-XX:MaxPermSize=4096m
-Dfile.encoding=UTF-8

Where I can find this settings in TAC server?

Thanks, Manuel

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 09:42:05 GMT

Hello,

Please add -Dfile.encoding=utf-8 parameter ,in commandLine start script file
start_cmdline.bat like:

cd /D "D:\Products\6.4\cmdline\studio" 
java -Xms64m -Xmx1024m -XX:MaxPermSize=256m -Dfile.encoding=utf-8 -jar plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 09:45:46 GMT

Hi, it's already written in <JETL_621>\cmdline\start_cmdline.bat

java -Xms256m -Xmx12288m -Dfile.encoding=utf-8 -jar plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar --disableShellInput -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 10:30:05 GMT

It should be on the Job. In TAC, in the Job, you should set the same parameters. You can also set JVM parameters on the job in TAC. There is a section to set JVM parameters. Make sure also that the individual components in the job is set to UTF8.

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 10:38:05 GMT

I've tried this way too, but the result is the same:

It's impossible!!!

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 11:14:26 GMT

The TAC does not execute the job. The TAC will send the job to the jobserver, and the jobserver agent will start a java.exe process with the JVM parameters provided. Not sure why it is not working. Can you provide the jobs?

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 13:01:27 GMT

@iburtally wrote:

Can you provide the jobs?

It's complicated because the input is a SAP call.

But I have add a tLogRow and I attached the studio log and the tac log.

You can see the differences, example:

Studio:

|303  |TR prel. a div.     |TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -> div.   |

TAC:

303|TR prel. a div.|TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -&#62; div.

The database column is a VARCHAR(20).

Could the error be due to the sap component as well as the mysql component?

Thanks for your help, Manuel

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 27 Jul 2017 15:38:26 GMT

@MZ wrote:
Could the error be due to the sap component as well as the mysql component?

OMG

Ok the problem is on the sap component because also eliminating the tMysqlOutput component the encoding is wrong on TAC.

Studio:

|303  |TR prel. a div.     |TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -> div.   |

TAC:

303|TR prel. a div.|TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -&#62; div.

What happens?

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Fri, 28 Jul 2017 03:52:02 GMT

Hello,

With your subscription solution, would you mind creating a case on talend support portal so that we can give you a remote assistance(webex) on your issue through support cycle with priority.

Please log on to the Support portal https://login.talend.com/support-login.php

Many thanks.

Best regards

Sabrina

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Fri, 28 Jul 2017 18:29:45 GMT

I think that this Jira may be related: https://jira.talendforge.org/browse/TDI-35499

It seems to have been corrected in Studio from 6.2.1, but it's possible we may not have patched the Job Server. We will look into it.

Thanks,

Arthur

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Wed, 02 Aug 2017 08:44:25 GMT

Hi all, with the support team we realized that the cmdline is executed by default in 32-bit.

We modified the commandline.bat file to run the 64-bit cmdline.

Thank you.

Manuel

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Anonymous — Thu, 03 Aug 2017 04:27:41 GMT

@MZ wrote:

Hi all, with the support team we realized that the cmdline is executed by default in 32-bit.

We modified the commandline.bat file to run the 64-bit cmdline.

Thank you.

Manuel

Hello Manuel,

Thanks for your feedback and sharing the solution here.

Best regards

Sabrina