Using dynamic schema throws error in tSnowflakeInput component

humapathy — Sat, 16 Nov 2024 06:39:21 GMT

Hi I have a very simple workflow to source data from snowflake and store it out as flat file. When I use dynamic as the schema type, the job fails with following error code if I explicitly specify the column names in my query. However, when i just specify 'Select *' it just runs fine even having 'dynamic' as the datatype.

INFO: sql command to get primary key metadata: show /* JDBCatabaseMetaData.getPrimaryKeys() */ primary keys in table "OPENFIT"."RECURLY_STITCH_PROD"."RECURLY_SUBSCRIPTIONS"

[FATAL]: bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow - tSnowflakeInput_2 2

java.lang.ArrayIndexOutOfBoundsException: 2

at org.talend.components.common.avro.JDBCResultSetIndexedRecordConverter$ResultSetIndexedRecord.get(JDBCResultSetIndexedRecordConverter.java:111)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.getDynamicValues(DiOutgoingDynamicSchemaEnforcer.java:146)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.get(DiOutgoingDynamicSchemaEnforcer.java:105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeInput_2Process(Openfit_Snowflake_Simple_Flow.java:1105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeConnection_2Process(Openfit_Snowflake_Simple_Flow.java:489)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.runJobInTOS(Openfit_Snowflake_Simple_Flow.java:1644)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.main(Openfit_Snowflake_Simple_Flow.java:1475)

[statistics] disconnected

Exception in component tSnowflakeInput_2 (Openfit_Snowflake_Simple_Flow)

java.lang.ArrayIndexOutOfBoundsException: 2

at org.talend.components.common.avro.JDBCResultSetIndexedRecordConverter$ResultSetIndexedRecord.get(JDBCResultSetIndexedRecordConverter.java:111)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.getDynamicValues(DiOutgoingDynamicSchemaEnforcer.java:146)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.get(DiOutgoingDynamicSchemaEnforcer.java:105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeInput_2Process(Openfit_Snowflake_Simple_Flow.java:1105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeConnection_2Process(Openfit_Snowflake_Simple_Flow.java:489)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.runJobInTOS(Openfit_Snowflake_Simple_Flow.java:1644)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.main(Openfit_Snowflake_Simple_Flow.java:1475)

Job Openfit_Snowflake_Simple_Flow ended at 11:24 06/02/2019. [exit code=1]

Re: Using dynamic schema throws error in tSnowflakeInput component

Anonymous — Thu, 28 Feb 2019 07:11:10 GMT

Hello,

Could you please clarify in which talend version/edition you are?

Best regards

Sabrina

Re: Using dynamic schema throws error in tSnowflakeInput component

humapathy — Thu, 28 Feb 2019 07:25:15 GMT

We are usin Talend Studio 6.4. Please note that this behavior is not consistent for all tables. For a different table in the same Schema, it works fine with dynamic Schema and select clause with explicit column names.

topic Using dynamic schema throws error in tSnowflakeInput component in Talend Studio

Using dynamic schema throws error in tSnowflakeInput component

Re: Using dynamic schema throws error in tSnowflakeInput component

Re: Using dynamic schema throws error in tSnowflakeInput component