Data Load Duration

Anonymous — Mon, 05 Nov 2018 13:52:19 GMT

Hello,

The requirement is to perform full data load everyday from Oracle to HDFS file system. The approx data volume is 20 M. Would like to know

1. How much time (min, Hrs ) will it take to load data

2. which is faster Talend Spark or Talend regular ETL

Thanks In Advance!

Re: Data Load Duration

David_Beaty — Tue, 06 Nov 2018 10:38:34 GMT

Hi Shanky,

There are a number of factors that would contribute to how much time it would take to load, like:

1/Source Data format (CSV, XML, JSON, etc), also number of source files (1 file of 20m rows or 20m files of 1 row).

2/Transformations required

3/End point server DB

4/Talend execution server spec & location (on-premise/cloud)

5/End point DB server spec & location (on-premise/cloud)

Even when you know all of these, actually defining an execution duration would be challenging. Sorry this reply hasn't been much help, but from the one fact you gave, it's just impossible to say.

Thanks

David

topic Re: Data Load Duration in Talend Studio

Data Load Duration

Re: Data Load Duration