Data Load Duration

[Anonymous]

Hello, 

The requirement is to perform full data load everyday from Oracle to HDFS file system. The approx data volume is 20 M. Would like to know 

1. How much time (min, Hrs ) will it take to load data 

2. which is faster Talend Spark or Talend regular ETL 

Thanks In Advance!

[David_Beaty]

Hi Shanky,

There are a number of factors that would contribute to how much time it would take to load, like:

1/Source Data format (CSV, XML, JSON, etc), also number of source files (1 file of 20m rows or 20m files of 1 row).

2/Transformations required

3/End point server DB

4/Talend execution server spec & location (on-premise/cloud)

5/End point DB server spec & location (on-premise/cloud)

Even when you know all of these, actually defining an execution duration would be challenging. Sorry this reply hasn't been much help, but from the one fact you gave, it's just impossible to say.

Thanks

David