Problem with insertion in Oracle

[leonchacal]

I have a csv file with more or less 15k lines and when I use talend to load this file into an Oracle table, talend takes 10 ~ 12 minutes. If I use SqlDeveloper the time to load the file is 30 ~ 45 seconds.

Do I need to configure something in Talend so that it has the same performance as Developer?

[Anonymous]

Hello,

What's your job row rate? What does your job design look like? Are you using Oracle bulk component?

Generally speaking, the followings aspects could affect the job performance:

1. The volume of data, read a large of data set, the performance will degrade

2. The struture of data, if there are so many columns on tOracleInput, it will consume many memory and much time for transferring the data during the job execution.

3. The database connection, the job always runs better if the database is installed on local, if the database is on another machine, even you are on VPN, you may have the congestion and latency issues.

Best regards

Sabrina

[leonchacal]

I'm using tOracleOutput and the row rate is approximately 25 rows per second.

I understood your points, Sabrina, but what intrigues me is that using SQL Developer, the performance of loading the same files, with the same columns, using the VPN is much faster than what Talend can do.

In this image you can see the comparison between SQL Developer and Talend.

While SQL Developer took 30 seconds to upload the file to the table, Talend took 515 seconds to load the same file.

[Anonymous]

HI

1- Creating the DB connection on tOracleOutput instead of using an existing DB connection will have a better performance, have a try.

2- There are many components in the job, all the data processing and data transfer take times.

Regards

Shong

[leonchacal]

I am already using an existing connection in tOracleOutput.

Even if I create a job with only the connection component and tOracleOutput, the performance remains the same, very low

[Anonymous]

If database server and Talend Studio are on different machine, probably a limitation of the network, refer to this topic.

https://community.talend.com/s/feed/0D53p00007vCoLcCAK?language=en_US