Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello community,
I am quite new to talend, creating jobs has been fairly easy. However i ran into a persistent dependency crash when using the standard tBigQueryOutput component to stream data out to Google BigQuery.
tDBConnection $\rightarrow$ tDBInput $\rightarrow$ tLogRow $\rightarrow$ tBigQueryOutput
The source database connection and extraction phases work perfectly. The rows print out to the console via tLogRow right up to the crash.
The component successfully handles the initial phase by creating and populating the staging file locally (86 KB size verified on disk).
The exact millisecond the job tries to instantiate the Google client to transition the data to the cloud, the job throws a ClassNotFoundException.
I have checked the Modules view inside Talend Studio, but google-api-core is entirely absent from the platform list and local artifact repository database. But others like google-cloud-core , google-api-client, google-api-services, google-cloud-bigquery, google-cloud-storageare present.
Interestingly, I have another job in the exact same workspace that runs perfectly without any dependency errors. Its layout is:
tBigQuerySQLRow $\rightarrow$ tLogRow $\rightarrow$ tBigQueryOutput
This suggests the required Google jar files may be present in my Talend environment, but for some reason, the specific job fetching data from an external tDBInput database source fails to compile/map the google-api-core library at runtime.
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/api/core/CurrentMillisClock
at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:322)
at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:91)
at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:30)
at com.google.cloud.bigquery.BigQueryOptions$Builder.build(BigQueryOptions.java:86)
at talend_tmc_project.get_invoice_debug_ver_0_1.get_invoice_debug_ver$1BigQueryBuilder_tBigQueryOutput_1.newBigQuery(get_invoice_debug_ver.java:5734)
at talend_tmc_project.get_invoice_debug_ver_0_1.get_invoice_debug_ver.tDBInput_1Process(get_invoice_debug_ver.java:5738)
at talend_tmc_project.get_invoice_debug_ver_0_1.get_invoice_debug_ver.tDBConnection_1Process(get_invoice_debug_ver.java:819)
at talend_tmc_project.get_invoice_debug_ver_0_1.get_invoice_debug_ver.runJobInTOS(get_invoice_debug_ver.java:6908)
at talend_tmc_project.get_invoice_debug_ver_0_1.get_invoice_debug_ver.main(get_invoice_debug_ver.java:6417)
Caused by: java.lang.ClassNotFoundException: com.google.api.core.CurrentMillisClock
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 9 more
Cleaned the project and forced code re-compilation.
Attempted to manually load dependencies using tLibraryLoad.
Verified that the underlying jar files are working in BigQuery-only isolated jobs.
How can I force Talend Studio to rebuild or correctly share the foundational google-api-core library dependencies with this database-driven job? Is there a project-level setting or index file that needs to be reset?
Talend version
Talend Cloud API Services Platform
Version: 8
Build id: 20260102_0846-patch
License Expiration date: 11/29/2030
(c) Copyright 2006-2024 Talend. All rights reserved.
Visit http://www.qlik.com
Thank you in advance for your assistance! Attached are also screenshots of installed modules
Hello,
What librairies did you load using tLibrartyLoad ?
If not already tested , could you add a component "tLibraryLoad" and load "api-common-2.0.0.jar" ?
Kind regards
Denis
Hello,
What librairies did you load using tLibrartyLoad ?
If not already tested , could you add a component "tLibraryLoad" and load "api-common-2.0.0.jar" ?
Kind regards
Denis
Hello Denis, this works thank you