topic Re: Issue when Using 2 tBigQueryOutput in 1 job. in Talend Studio

Issue when Using 2 tBigQueryOutput in 1 job.

Anonymous — Sat, 16 Nov 2024 06:16:56 GMT

Hi all.

I have following failing scenario:

Sources: File and BQ table

ETL: Join and Filter

Target: BQ table1 and BQ table2.

It fails with error:

org.talend.designer.runprocess.ProcessorException: Job compile errors
At least job "BigQueryFileJoinToBQAndBQ" has a compile errors, please fix and export again.
Error Line: 2496
Detail Message: Duplicate local variable tableNotExist

I have also following passing scenario:

Sources: File and BQ table

ETL: Join and Filter

Target: BQ table1 and File.

so the question, does any body used 2 tBigQueryOutput in 1 job ?

and is it working ?

I am using Talend BD Studio : TOS_BD-20181026_1147-V7.1.1

Re: Issue when Using 2 tBigQueryOutput in 1 job.

Anonymous — Fri, 22 Mar 2019 08:05:30 GMT

Hello @ssomanchi ,

Could you please check your "Code" tab in your job? There will be your compile error highlighted in red line. Could you post it here, please?

Best regards

Sabrina

Re: Issue when Using 2 tBigQueryOutput in 1 job.

vapukov — Fri, 22 Mar 2019 09:41:26 GMT

Hi,

with versions 7.* - it looks like a global "pattern error" for several not often used components

we have already registered 3 tickets for different components for the subscription version (at least 1 resolved)

what to do with Open Studio? I just hope Talend team will fix it in the feature version

as a bypass - split the job into 2 separate subjobs (in the same job).

For example - store data into 2 local csv file and then 1 by 1 transfer them to BigQuery

it little ugly, but Community version - don't mean support between releases

regards, Vlad

Re: Issue when Using 2 tBigQueryOutput in 1 job.

Anonymous — Fri, 22 Mar 2019 10:37:04 GMT

Thanks Sabrina

Thanks Vapukov. - Thanks for suggestion, It is not critical, Since its a evaluation. I am just trying to see the functionality and limitations.

here is the code :

boolean tableNotExist = table_tBigQueryOutput_1_tBQBE == null;
if (false || tableNotExist) {
boolean typeSupported = true;
java.util.List<com.google.cloud.bigquery.Field> fields_tBigQueryOutput_1_tBQBE = new java.util.ArrayList<>();

if (!typeSupported) {
throw new IllegalArgumentException(
"unsupported type for column [id_]: "
+ "id_String");
}
com.google.cloud.bigquery.Field field_0 = com.google.cloud.bigquery.Field
.of("id_",
com.google.cloud.bigquery.LegacySQLTypeName.STRING);
fields_tBigQueryOutput_1_tBQBE.add(field_0);

if (!typeSupported) {
throw new IllegalArgumentException(
"unsupported type for column [date_]: "
+ "id_String");
}

Re: Issue when Using 2 tBigQueryOutput in 1 job.

Anonymous — Fri, 29 Mar 2019 11:17:33 GMT

Hello,

I see the that the bug is marked as rejected because it's duplicated with TBD-8102.

Do you know when this will be fixed? The 7.0.X versions are affected?

We are depending on this to choose this ETL for the company or not also...

Re: Issue when Using 2 tBigQueryOutput in 1 job.

Anonymous — Fri, 29 Mar 2019 12:25:08 GMT

Splitting the job in subjobs inside the same job won't work...