topic Re: [resolved] tOracleInput to tOracleOutput very slow in Talend Studio

[resolved] tOracleInput to tOracleOutput very slow

Anonymous — Tue, 08 Apr 2014 09:20:53 GMT

Hi folks,
I'm suffering from the problem that a simple copy from one table to another is very slow. The tables are on different schemas but on the same database. The Oracle Input is a select-query with a simple WHERE clause.
I'm connected to the database server via VPN. Maybe this is the limiting factor? But still the operation should be faster than ~5 rows per second....
See attached Screenshot.

Re: [resolved] tOracleInput to tOracleOutput very slow

Anonymous — Tue, 08 Apr 2014 16:24:35 GMT

assuming your database is healthy this is probably a limitation of the network throughput between the database and where talend is running. Some options are to avoid transferring data across the network are:
* tOracleRow - if you can write your process as a single sql statement ( insert into ... select from )
* tOracleSP - put your process in a stored procedure and call it
* use ELT components - tELTOracleInput, tELTOracleMap, tELTOracleMap
there is a bit of a learning curve with the ELT components as they work quite differently than other components. This is because they primarily for generating SQL.

Re: [resolved] tOracleInput to tOracleOutput very slow

Anonymous — Tue, 08 Apr 2014 16:29:10 GMT

assuming your database is healthy this is probably a limitation of the network throughput between the database and where talend is running. Some options are to avoid transferring data across the network are:
* tOracleRow - if you can write your process as a single sql statement ( insert into ... select from )
* tOracleSP - put your process in a stored procedure and call it
* use ELT components - tELTOracleInput, tELTOracleMap, tELTOracleMap
there is a bit of a learning curve with the ELT components as they work quite differently than other components. This is because they primarily for generating SQL.

Mike, you are right, I run the job at a machine which is on the same subnet as the database and the throughput was much better.
I already tried ELT but gave up to quick maybe ^^ Will take a look at it again, thanks anyway!

Re: [resolved] tOracleInput to tOracleOutput very slow

willm1 — Tue, 08 Apr 2014 16:35:21 GMT

In addition to that, check your jvm settings for your Studio - they may be low.... https://help.talend.com/search/all?query=Allocating+more+memory+to+the+Talend+Studio&content-lang=en