topic Re: Unrecognized type id_byte[] in Talend Studio

Unrecognized type id_byte[]

Manoj_MUTHURAJA — Sat, 16 Nov 2024 02:12:20 GMT

Hello,

I am getting this error "Unrecognized type id_byte[] " .I am copying tables from one db to another db.I didn't had issues with most of the tables.With this error I couldn't find the error comes where .I tried changing all the datatype to varchar but still it is not accepting.I don't have any idea of this error.

Thanks for suggestions if you have any idea,

Regards,

Manoj

Re: Unrecognized type id_byte[]

Anonymous — Tue, 09 Jun 2020 05:17:56 GMT

Hi
Which database? Do you use dynamic schema?

Regards
Shong

Re: Unrecognized type id_byte[]

Manoj_MUTHURAJA — Tue, 09 Jun 2020 06:47:51 GMT

Hi,

I'm copying data of tables from Oracle to Snowflake.Just the tables are dynamic , schema remains constant.

There is a column of datatype byte[] on a table of oracle.But generally I use the datatype dynamic for all tables it is working ,but I don't know why it is not accepting the datatype byte[].

If I copy the table manually it means one to one in another job,there are no issues.I think maybe on the snowflake side it couldn't identify the datatype (byte[]).

This column in the database is of datatype Raw(16) which is recogonized as byte[] in talend.

Regards,

Manoj

Re: Unrecognized type id_byte[]

Anonymous — Wed, 10 Jun 2020 04:13:57 GMT

Hi
I have seen a similar issue, see below.

Exception in component tSnowflakeOutput_1 (testJob)

java.lang.UnsupportedOperationException: Unrecognized type id_Object

at org.talend.daikon.di.DiIncomingSchemaEnforcer.diToAvro(DiIncomingSchemaEnforcer.java:199)

at org.talend.daikon.di.DiIncomingSchemaEnforcer.addDynamicField(DiIncomingSchemaEnforcer.java:149)

at local_project651.testjob_0_1.testJob.tMSSqlInput_1Process(testJob.java:1220)

at local_project651.testjob_0_1.testJob.runJobInTOS(testJob.java:2728)

at local_project651.testjob_0_1.testJob.main(testJob.java:2541)

Root Cause
The above error was due to the dbType in the mapping files are not compatible. For example of the test job for the error above; the job retrieve data from the table name account1 in Microsoft Server database then insert data into SnowFlake database. Both tables in both database has the same columns and the same datatype (note: datatype is not the same as dbType) in the schema.

When customer used the dynamic schema in their job to retrieve data from the dabase (ex: SQL Server in this case) then insert data into SnowFlake with dynamic schema. The dbType of input columns in the job (ex: tMSSqlInput) will be rely on the setting in the Talend Type Mapping Files. For this case, it is the mapping_MSSQL.xml. For the dbType of Money in the mapping_MSSQL.xml file for acct_balance column; the setting is only has

<dbType type="MONEY">

<talendType type="id_Object" default="true" />

</dbType>

But in the mapping_Snowflake.xml, there is no dbType of MONEY. It only has the type (which is datatype) of Double:

<dbType type="DOUBLE">

<talendType type="id_Double" default="true" />

</dbType>

Therefore, when it passes data from the tMSSqlInput component to tSnowflakeOutput component, it is error out in the tSnowflakeOutput component for not recognize the id_Object.

Solution

The easy way to fix this error is, we need to change the dbType from id_Object to id_Double for the MONEY in the mapping_MSSQL.xml file. So, it is compatible to the mapping_Snowflake.xml. To do this by follow below steps:

Open the studio
Navigate to File
Click on Edit Project properties
Expand the General
Click on the Metadata of TalendType
Scroll down then click on the mapping_MSSQL.xml and then click Edit
Scroll down the mapping_MSSQL.xml file to the tag <dbType type="MONEY">
<dbType type="MONEY">

<talendType type="id_Object" default="true" />

</dbType>

Then change from <talendType type="id_Object" default="true" /> in the tag <dbType type="MONEY"> to <talendType type="id_Double" default="true" />
Example:

From:

<dbType type="MONEY">

<talendType type="id_Object" default="true" />

</dbType>

To:

<dbType type="MONEY">

<talendType type="id_Double" default="true" />

</dbType>

After, made the above change then ran the job; the job should works.

Hope it helps you!

Regards
Shong

Re: Unrecognized type id_byte[]

Manoj_MUTHURAJA — Wed, 10 Jun 2020 07:26:15 GMT

Hi Shong,

Thanks for your reply.It is really a good solution.I have found another solution that as we have a mass volume of tables and datas,the tables which causes the problem we will choose just the columns which are not causing the problem.

We have an excel sheet which has the list of tables with the columns and the where conditions.

So in the conditions I will eliminate the columns which has the issues.For now its okay I guess ,but If the column which has an issue is important then I have to go through the steps you have proposed.Thank you very much,your reply is very clear,I hope it will be useful for me in future.

Have a good day,

Manoj

Re: Unrecognized type id_byte[]

Anonymous — Wed, 10 Jun 2020 07:32:37 GMT

Thank you for your positive feedback, and it’s great to see you get help from the community 😉

Re: Unrecognized type id_byte[]

SIyer1613043172 — Mon, 09 Aug 2021 14:28:37 GMT

Hi Shong,

I'm having the same issue while using dynamic schema for varbinary columns in the source. My source is SAP Hana and target is Snowflake. Tried modifying the mapping xmls but doesnt seem to work. Getting a Unrecognized type id_byte[] error.

Any advice would be highly appreciated

Re: Unrecognized type id_byte[]

KB_Parpally — Fri, 20 Jan 2023 19:59:30 GMT

Hello,

I am also facing the similar error, could you please help me you have find any solution?

[FATAL] 14:56:22 eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf- tDBOutput_1 Unrecognized type id_Object

java.lang.UnsupportedOperationException: Unrecognized type id_Object

at org.talend.codegen.enforcer.IncomingSchemaEnforcer.diToAvro(IncomingSchemaEnforcer.java:306) ~[talend-codegen-utils-0.31.0.jar:?]

at org.talend.codegen.enforcer.IncomingSchemaEnforcer.addDynamicField(IncomingSchemaEnforcer.java:173) ~[talend-codegen-utils-0.31.0.jar:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBInput_1Process(jobitemmastercdstosf.java:4090) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBInput_3Process(jobitemmastercdstosf.java:2675) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_6Process(jobitemmastercdstosf.java:1491) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBConnection_2Process(jobitemmastercdstosf.java:1353) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_14Process(jobitemmastercdstosf.java:4855) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBConnection_3Process(jobitemmastercdstosf.java:4715) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tRESTClient_3Process(jobitemmastercdstosf.java:6860) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_3Process(jobitemmastercdstosf.java:5934) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tFileInputDelimited_1Process(jobitemmastercdstosf.java:5774) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.runJobInTOS(jobitemmastercdstosf.java:8210) [classes/:?]

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.main(jobitemmastercdstosf.java:7670) [classes/:?]

Exception in component tDBOutput_1 (jobitemmastercdstosf)

java.lang.UnsupportedOperationException: Unrecognized type id_Object

at org.talend.codegen.enforcer.IncomingSchemaEnforcer.diToAvro(IncomingSchemaEnforcer.java:306)

at org.talend.codegen.enforcer.IncomingSchemaEnforcer.addDynamicField(IncomingSchemaEnforcer.java:173)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBInput_1Process(jobitemmastercdstosf.java:4090)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBInput_3Process(jobitemmastercdstosf.java:2675)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_6Process(jobitemmastercdstosf.java:1491)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBConnection_2Process(jobitemmastercdstosf.java:1353)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_14Process(jobitemmastercdstosf.java:4855)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tDBConnection_3Process(jobitemmastercdstosf.java:4715)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tRESTClient_3Process(jobitemmastercdstosf.java:6860)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tJava_3Process(jobitemmastercdstosf.java:5934)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.tFileInputDelimited_1Process(jobitemmastercdstosf.java:5774)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.runJobInTOS(jobitemmastercdstosf.java:8210)

at eim_itemmaster.jobitemmastercdstosf_0_1.jobitemmastercdstosf.main(jobitemmastercdstosf.java:7670)

job ended : 2023-01-20 14:56:22

[statistics] disconnected