Data load performance

[Anonymous]

Hi All,

I have a talend job where I am loading data from a oracle source to a greenplum table. I have used tMap in between and the source row count is around 72 million. I have enabled parallel execution and the cursor from the source is also enabled. But still the data loading speed is not upto the mark and it's taking a lot of time to even load 10 million and then the connection is timing out. Could anyone please let me know how can I speed up the performance?

Thanks,

Dyutiman

[vapukov]

test job with - tGreenplumBulkExec

[Anonymous]

Does this component load a table or file..Because we are trying to do a on to one load from one table to another

[vapukov]

as I remember:

I am loading data from a oracle source to a greenplum table

You are copy data from one database to another database

and this component do this by 2 steps:

- export data to csv file

- run bulk insert command to import csv file

this combination work much faster in most of cases