topic Re: Reusing tDBConneciton over mutliple Hosts dynamically. in Talend Studio

Reusing tDBConnection over mutliple Hosts dynamically.

DougIrwin — Mon, 01 Sep 2025 02:58:07 GMT

Hi all,

New to Talend but I've been around a long time with data and coding.

I need to pull data from a bunch of servers that have a consistent database and table name structure. I've set up a basic tDBConnection->tDBInput->tLogRow->tDBClose prototype that works fine.

Then I've added a tJava at the very beginning to try to get the Connection as an object, with a view to updating the Host in the Connection dynamically. The aim is to change this to some kind of Iterator, once I've proven this can work.

Unfortunately I'm not having any luck getting the Connection in the tJava, which throws this at runtime:

Starting job ConnectionTest at 14:38 28/08/2025.
[statistics] connecting to socket on port 3689
[statistics] connected
Exception in component tJava_1 (ConnectionTest)
java.lang.NullPointerException: Cannot invoke "java.sql.Connection.getMetaData()" because "c" is null
at xxx.connectiontest_0_1.ConnectionTest.tJava_1Process(ConnectionTest.java:552)
at xxx.connectiontest_0_1.ConnectionTest.runJobInTOS(ConnectionTest.java:2397)
at xxx.connectiontest_0_1.ConnectionTest.main(ConnectionTest.java:2035)
[FATAL] 14:38:56 xxx.connectiontest_0_1.ConnectionTest- tJava_1 Cannot invoke "java.sql.Connection.getMetaData()" because "c" is null
java.lang.NullPointerException: Cannot invoke "java.sql.Connection.getMetaData()" because "c" is null
at xxx.connectiontest_0_1.ConnectionTest.tJava_1Process(ConnectionTest.java:552) [classes/:?]
at xxx.connectiontest_0_1.ConnectionTest.runJobInTOS(ConnectionTest.java:2397) [classes/:?]
at xxx.connectiontest_0_1.ConnectionTest.main(ConnectionTest.java:2035) [classes/:?]
[statistics] disconnected

Job ConnectionTest ended at 14:38 28/08/2025. [Exit code = 1]

See attachment for what I have, which is working so long as the tJava is disabled.

Any suggestions would be gratefully received.

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

Denis_Segard — Thu, 28 Aug 2025 05:49:53 GMT

Hello,

You may define the Host as a context variable and set this context variable in a tJava or any other way ..

Kind regards
Denis

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

DougIrwin — Thu, 28 Aug 2025 05:58:15 GMT

Hey Denis!

So I got the email about your reply and created a context.hostName, setting it to an initial value and updating my tDBConneciton object to use that. Running that worked, so I modified the tJava to change context.hostName to a different server. That worked perfectly!

Thanks so much for the tip! Now to get an iterator and list and etc, etc 🙂

Thanks again!

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

DougIrwin — Thu, 28 Aug 2025 23:13:53 GMT

One thing I just noticed with this solution is that it doesn't work well with parallel execution - I assume because context variables are global. Does anyone know if there's a way to do this that would work with parallel execution?

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

Dave_Simo — Fri, 29 Aug 2025 12:47:24 GMT

Hello @DougIrwin

I don't know how you set up your parallelism, but try this design maybe it will help you.

With 2 parallel executions for example.

Best Regards

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

DougIrwin — Mon, 01 Sep 2025 01:19:12 GMT

I seem to be missing something. How is your tFixedFlowInput_2 picking up the data from tFlowToIterate_1 to display in the tLogRow ??

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

DougIrwin — Mon, 01 Sep 2025 03:00:33 GMT

Ah looks like you "bind" the row1.values in tFixedFlowInput_2. At least that works for me. Not sure if I can then use this in a tDBConnection but I'll have a play with it. Thanks for the suggestion!

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

Dave_Simo — Mon, 01 Sep 2025 07:19:27 GMT

Hello @DougIrwin

Yes in tFixedFlowInput_2 I linked it with row1.values.

And as input I have this in tFixedFlowInput_1

Best Regards

Re: Reusing tDBConneciton over mutliple Hosts dynamically.

DougIrwin — Mon, 01 Sep 2025 23:19:06 GMT

Thanks @Dave_Simo that's essentially the same as what I have - the only difference being my column names. Works a charm, thanks!