Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik GA: Multivariate Time Series in Qlik Predict: Get Details
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Slow performance on insert when using tOracleConnection

I've noticed a significant performance issue when I use tOracleConnection component to insert data. My environment is Windows XP, Talend IS 3.2.2, with an Oracle database. Here's the scenario:
Use tOracleConnection component to establish a connection to the database.
Create tOracleInput component using the existing tOracleConnection.
Map data into tOracleOutput component using the existing tOracleConnection.
I find that the data will flow into the tOracleOutput component extremely slowly using this technique (approximately 2-4 rows per second).
However, if I clear the "Use Existing Connection" checkbox on the tOracleOutput component and use either a repository connection or a built it connection the same program processes data much faster (several thousand rows per second).
Can someone explain to me why I experience such a performance hit when using an existing connection for the Output component? (I don't notice the same issue with the input component.
Thanks in advance.
Joe
Labels (3)
14 Replies
Anonymous
Not applicable
Author

I use the tOracleConnection for lookups with the parameter "use cursor". And for the Output (tOracleOuput), i use a direct connection (without tOracleConnection), but configured by context, and the parameter "advances settings>Commit every" configured to 10000, performances are better.
Can you try ?
Anonymous
Not applicable
Author

Yes shalouf, I've tried your suggestion in the past and had the same results that you are describing. The problem is that we should be able to use a tOracleConnection on the tOracleOutput. In your situation, you're requiring the Talend job to create a second connection to the database, which should not be required. The tOracleConnection component is intended to prevent the need for creating a new connection with each component. We should be able to use this component without experiencing performance degradation.
Anonymous
Not applicable
Author

Anonymous
Not applicable
Author

I did also notice slow performance and activated "parallel execution". This seriously increases performance! However, I'm having two problems, please see my post here: https://community.talend.com/t5/Design-and-Development/Parallel-execution-tOracleOutput/td-p/91778
_AnonymousUser
Specialist III
Specialist III

Yes shalouf, I've tried your suggestion in the past and had the same results that you are describing. The problem is that we should be able to use a tOracleConnection on the tOracleOutput. In your situation, you're requiring the Talend job to create a second connection to the database, which should not be required. The tOracleConnection component is intended to prevent the need for creating a new connection with each component. We should be able to use this component without experiencing performance degradation.

Absolutely, whenever you check Use Existing Connection , immediatly Commit Every and Batch Size options will be vanished. this should not be the case.