topic Re: Migrating 2 billion records to SAP HANA in Talend Studio

Migrating 2 billion records to SAP HANA

karandama2006 — Sat, 16 Nov 2024 06:43:22 GMT

Hi ,

I'm trying to Migrate 2 billion records from a table in SQL Server to a table in SAP HANA using talend.

Current throughput which I'm getting is close to 25k rows per second , with this speed it would take more than a day to successfully load that data .

Is there a smart way of doing this ?

Thanks

Re: Migrating 2 billion records to SAP HANA

vapukov — Thu, 31 Jan 2019 00:27:34 GMT

Hi,

I haven't so big data just for test with Hana, but for most of the databases - build in method for bulk load data fom csv is fastest from all possible

HANA support it as well

but also speed depends from the sources and transformations as well - do you request data from a database? is it contain any order operations? and etc.

Re: Migrating 2 billion records to SAP HANA

karandama2006 — Sat, 23 Feb 2019 13:32:44 GMT

Yes , Data is requested from Database , there is no transformation except adding a timestamp