Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
leonchacal
Contributor
Contributor

Problem with insertion in Oracle

I have a csv file with more or less 15k lines and when I use talend to load this file into an Oracle table, talend takes 10 ~ 12 minutes. If I use SqlDeveloper the time to load the file is 30 ~ 45 seconds.

Do I need to configure something in Talend so that it has the same performance as Developer?

Labels (4)
5 Replies
Anonymous
Not applicable

Hello,

What's your job row rate? What does your job design look like? Are you using Oracle bulk component?

Generally speaking, the followings aspects could affect the job performance:

1. The volume of data, read a large of data set, the performance will degrade

2. The struture of data, if there are so many columns on tOracleInput, it will consume many memory and much time for transferring the data during the job execution.

3. The database connection, the job always runs better if the database is installed on local, if the database is on another machine, even you are on VPN, you may have the congestion and latency issues.

Best regards

Sabrina

 

leonchacal
Contributor
Contributor
Author

I'm using tOracleOutput and the row rate is approximately 25 rows per second.

I understood your points, Sabrina, but what intrigues me is that using SQL Developer, the performance of loading the same files, with the same columns, using the VPN is much faster than what Talend can do.

In this image you can see the comparison between SQL Developer and Talend.

While SQL Developer took 30 seconds to upload the file to the table, Talend took 515 seconds to load the same file.

 

069U1000001mLj9IAE.png

Anonymous
Not applicable

HI

1- Creating the DB connection on tOracleOutput instead of using an existing DB connection will have a better performance, have a try.

2- There are many components in the job, all the data processing and data transfer take times.

 

Regards

Shong

leonchacal
Contributor
Contributor
Author

I am already using an existing connection in tOracleOutput.

Even if I create a job with only the connection component and tOracleOutput, the performance remains the same, very low

 

Anonymous
Not applicable

If database server and Talend Studio are on different machine, probably a limitation of the network, refer to this topic.

https://community.talend.com/s/feed/0D53p00007vCoLcCAK?language=en_US