topic Re: Data load performance in Talend Studio

Data load performance

Anonymous — Thu, 12 Jul 2018 22:30:04 GMT

Hi All,

I have a talend job where I am loading data from a oracle source to a greenplum table. I have used tMap in between and the source row count is around 72 million. I have enabled parallel execution and the cursor from the source is also enabled. But still the data loading speed is not upto the mark and it's taking a lot of time to even load 10 million and then the connection is timing out. Could anyone please let me know how can I speed up the performance?

Thanks,

Dyutiman

Re: Data load performance

vapukov — Fri, 13 Jul 2018 01:40:22 GMT

test job with - tGreenplumBulkExec

Re: Data load performance

Anonymous — Fri, 13 Jul 2018 05:36:56 GMT

Does this component load a table or file..Because we are trying to do a on to one load from one table to another

Re: Data load performance

vapukov — Fri, 13 Jul 2018 13:27:55 GMT

as I remember:

I am loading data from a oracle source to a greenplum table

You are copy data from one database to another database

and this component do this by 2 steps:

- export data to csv file

- run bulk insert command to import csv file

this combination work much faster in most of cases