topic tDBConnection to Snowflake error; no connection in Talend Studio

tDBConnection to Snowflake error; no connection

Anonymous — Sat, 16 Nov 2024 03:48:14 GMT

Hi all,

Currently I am trying to connect in a tPrejob with a Snowflake database. After the prejob i have a tRESTRequest component and after the API call i am trying to access the Snowflake database and read data from it based on parameters from the API call. Job looks like this: tPrejob --> tDBConnection then tRESTRequest --> tFlowToIterate --> tDBInput....

The last part (reading data from Snowflake) works perfectly if i use the connection details from the metadata repository in the tDBInput component, so without the connection being made in the prejob. (i need it in the prejob so that the connection to Snowflake stays alive and it will not connect to the database on every call that is made to the API)

When i use only the prejob --> tDBConnection (Snowflake connection) and after that tDBInput (Snowflake read) --> tLogRow it also works perfectly! So it seems to have a problem when the tRESTRequest component is in between.

Does anyone have a solution for this?

PS. the error message is:

Exception in component tDBInput_1 (API_transactions)
java.lang.RuntimeException: Referenced component: tDBConnection_2 not connected
	at cloud_sandbox.api_transactions_0_1.API_transactions.tRESTRequest_1_LoopProcess(API_transactions.java:6234)...

Re: tDBConnection to Snowflake error; no connection

Anonymous — Tue, 17 Dec 2019 16:34:18 GMT

Hi,

Could you please add job and component screenshots for better clarity of your use case?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tDBConnection to Snowflake error; no connection

Anonymous — Tue, 17 Dec 2019 18:22:04 GMT

Yeah sure, here you are:

The components work perfect if i don't use the connection that is established in the prejob, but when i switch in the tDBInput(Snowflake) to the component next to the tPreJob then it fails and i get the error message that i have posted earlier.

PS. don't mind the tReplicate and tJava, those are just for logging and testing during development.

Re: tDBConnection to Snowflake error; no connection

Anonymous — Tue, 17 Dec 2019 19:02:53 GMT

Hi,

Could you please try the same flow but instead of tPreJob, use only tSnowflakeConnection and then connect to RESTRequest component using an On SubJob Ok link.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tDBConnection to Snowflake error; no connection

Anonymous — Tue, 17 Dec 2019 21:35:40 GMT

Hi Nikhil,

That would work but that is not a solution for my problem. Because i wan't to keep the connection open during the job time and i only want to open the connection to Snowflake once and then for every call use that open connection from performance perspective.

If i use the on subjob ok then the connection wil re-open each time a call is being sent. Please correct me if i'm wrong!

And as you can see, on the top right the 2 seperate components do actually work with the prejob + connection. it seems to have a problem with either the tRESTRequest component or the tFlowToIterate....

Re: tDBConnection to Snowflake error; no connection

Anonymous — Wed, 18 Dec 2019 02:52:36 GMT

Hi,

I am trying to understand whether there is any bug when the code is getting generated in your specific use case. If you are using Connection before tRest component, the connection will be established only once and it will happen before executing tRest component.

Could you please try and let me know the results along with job screenshots?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tDBConnection to Snowflake error; no connection

Anonymous — Wed, 18 Dec 2019 08:43:48 GMT

Hi Nikhil,

I have tried what you suggested but i am getting the same error, see the screenshot for details. Can you please provide more suggestions on how to fix this?

Also one thing i have tried in the meantime: i have replaced the tDBInput(Snowflake) with a tDBInput(MySQL) connection and that works perfectly, with the exact same setup as below. So the problem seems to occur only when having a connection to Snowflake...

Re: tDBConnection to Snowflake error; no connection

Anonymous — Wed, 18 Dec 2019 14:03:23 GMT

Hi,

I guess its a bug in the Product. If you are an enterprise customer, could you please raise a support case so that they will help to check with R&D team about this issue. Please add the community link also so that you do not have to repeat the same steps again.

If you are using Talend open source version, could you please create a JIRA ticket using below link?

https://jira.talendforge.org/

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved