topic Re: How to limit the job threads in Talend Studio

How to limit the job threads

Anonymous — Thu, 10 Apr 2014 13:24:18 GMT

Dear All,
We have a DI Job as a Web Service that calls several sub-jobs one after another.
The nature of a Web Service is its asynchronism and the Web Service client can start
any amount of web service requests one after another with the fatal consequences that
endless DI-Job-Threads come into being. Through this we get a problem with the database
connections:
java.sql.SQLException RA-01000: maximum open cursors exceeded
There are two choices to overcome this issue:
First we limit the threads of DI Jobs, but how can one web service request
make dormant and start again as soon as the number of running threads is
less than the maximum permitted threads.
Second all threads have access to a database connection pool.
I'd be very appreciated for any hints and help.
Kind regards
Hilderich

Re: How to limit the job threads

Anonymous — Thu, 10 Apr 2014 13:55:25 GMT

Hi Helderich,
Very good explanation is available at
http://stackoverflow.com/questions/12192592/java-sql-sqlexception-ora-01000-maximum-open-cursors-exceeded.
I think if we close the connection for respective DI job thread within himself after completion, we may not run into such problem.
How you are managing database connection in each DI job?
Thanks
Vaibhav

Re: How to limit the job threads

Anonymous — Thu, 10 Apr 2014 14:25:44 GMT

Hello sanvaibhav,
Thanks for your reply. In each (sub)job that cares for the database we use a tJDBCConnection and have the option activated for using a shared connection, that is an already existing connection or create a new one with a special name.
There is no auto commit enabled, instead we use the component tCommit and there is the option disabled to close the connection afterwards.
The parent job that includes the web service itself has no tJDBCConnection component and therefore it is not possible to close the connection after all other stuff has finished.
Kind regards
Hilderich

Re: How to limit the job threads

Anonymous — Fri, 11 Apr 2014 04:15:54 GMT

Hi,
What the build version are you using? Talend Open Source or Enterprise Subscription Version? Would you mind sharing your current job design screenshot into forum?
Best regards
Sabrina

Re: How to limit the job threads

Anonymous — Fri, 11 Apr 2014 05:27:45 GMT

Hi Helderich,
Following is something unexpected, something is wrong at some place or bug...
>>>
There is no auto commit enabled, instead we use the component tCommit and there is the option disabled to close the connection afterwards.
Thanks
Vaibhav

Re: How to limit the job threads

Anonymous — Fri, 11 Apr 2014 17:14:30 GMT

Hello All,
Now we follow another approach. We have removed the response from the beginning of the job and
instead give a response only when all stuff is done. But this isn't simple as it sounds because now
we have another problem with that approach. tLogCatcher in conjunction with tESBProviderFault
does not work:
https://community.talend.com/t5/Design-and-Development/tESBProviderFault-inside-tLogCatcher-doesn-t-work/td-p/94973
Kind Regards
Hilderich

Re: How to limit the job threads

Anonymous — Mon, 14 Apr 2014 04:45:46 GMT

Hi hilderich,
There is a response in https://community.talend.com/t5/Design-and-Development/tESBProviderFault-inside-tLogCatcher-doesn-t-work/td-p/94973, would you mind checking it?
Best regards
Sabrina