topic Re: how to control Parallel link per connection when using tOracleInput? in Talend Studio

how to control Parallel link per connection when using tOracleInput?

_AnonymousUser — Sat, 16 Nov 2024 10:46:33 GMT

hello,everybody:
I wonder know how to control Parallel link per connection when using tOracleInput since my table is very big and if just use one link per connect caused slow,What method can improve the efficiency of it ?
thanks very much!

Re: how to control Parallel link per connection when using tOracleInput?

_AnonymousUser — Sun, 28 Feb 2016 16:06:59 GMT

Frade wrote:
hello,everybody:
I wonder know how to control Parallel link per connection when using tOracleInput since my table is very big and if just use one link per connect caused slow,What method can improve the efficiency of it ?
thanks very much!

some details:
I need to use tOracleInput componets read the data of oracle table partition (1millions/10millions) to HDFS, 1milllions records cost 192s, and 10 millions records cost 29min, if full table, cost almost 3 Hours, check the connection detail in oracle, there just one connection and one parallel link to oracle, what's the way to improve the efficiency of it ?
if i use sqope 4 parallel link from oracle to HDFS, 1millions records just cost 64s, 10millions cost 7min.

Re: how to control Parallel link per connection when using tOracleInput?

Anonymous — Mon, 29 Feb 2016 08:27:35 GMT

Hi,
What's the product you are using? Open source or Subscription version?
Here is "Multi thread execution" feature available in open source. You can find it from job review-->Extra
Best regards
Sabrina