topic Re: Data Migration in Talend Studio

Data Migration

Anonymous — Mon, 03 Jul 2017 18:31:37 GMT

Hello

I want to Migrate my database from HP-UX 11i v2 to Redhat Linux Version 6.5.

I will like to know if i can use Talend to do this migration seamlessly and which product of Talend do i use

thanks

Buks

Re: Data Migration

Anonymous — Mon, 03 Jul 2017 18:45:26 GMT

Which database is it? Wouldn't it be easier to just do an export and import using the database tools?

Re: Data Migration

TRF — Mon, 03 Jul 2017 19:59:44 GMT

You don't us which db. If the source and target engine are the same, it would be simplier to use the export/import functionalities offered by the db.
But if you change the engine for example from Oracle to Postgresql, sure you can use Talend for this purpose.

Re: Data Migration

Anonymous — Mon, 03 Jul 2017 20:08:17 GMT

Sorry I forgot to mention the DB,

its from Oracle 11gR2 to Oracle 12c

By the way i want a seamless migration and minimize downtime since systems are in production

also data size is around 20TB

thanks

Re: Data Migration

Anonymous — Mon, 03 Jul 2017 20:11:37 GMT

Source database is Oracle 11g R2
Target database is Oracle 12c
size of data is around 20TB and I am thinking export will use a lot of system resources, and take a lot of time , i also want to avoid downtime

Re: Data Migration

TRF — Mon, 03 Jul 2017 20:55:49 GMT

Well, let say you've decided to go with Talend. Congrats, that's a good choice.
Now, because you probably have some rows to migrate, just take the time to design the best strategy for your case. Will you migrate 1 table after 1 or will you try to paralize some operations?
Also think about rejected records. Will you iterate until no error or will you restart from the beginning in case of error?
The last but not the least, will you directly connect to the source and target db at the same time, or will who export to flat files first, then import to the the target?
Talend is fairly flexible to let you find the best trail for your case. Don't hesitate to try, fail and try again.
And don't hesitate to share and/or ask questions.

Re: Data Migration

Anonymous — Mon, 03 Jul 2017 21:40:18 GMT

20TB data will take a long time. However, I still believe disk block copy is better than doing ETL data integration.

You can either:

- Use block copy replication and turn on replication first into a replica of the Oracle 11g. Then once the db is replicated, you temporarily turn off the replication, take of dump of the replicated db first as a backup, upgrade the Oracle 11g in place to Oracle 12g upgrading all your data files. This will take time but will not impact your production DB too much.

- Use Oracle DB link to link the Oracle 11g from the Oracle 12c. Then do ELT to get the data from 11g to 12c. This will impact your production DB if it is very heavy in processing.

- Use Talend to migrate table by table. Still it will take a long time to do so. This is a good operation if the target DB is in AWS for example. AWS has a service called snowball where you send them your 20TB disks and they load it up for you by literally plugging the disk into their datacenter. You won't upload 20TB. It will take ages unless your connection to internet to AWS is gigabits.

In all the above cases, you will want to be able to replay the log of the Oracle 11g database for the duration of the migration, after the migration is over, so as to bring the migrated db back to the same level as the original one if the original one keeps on running while migration is happening in the background

Re: Data Migration

Anonymous — Tue, 04 Jul 2017 00:11:31 GMT

Hi, I have not concluded to go with Talend , i am looking for the best option to minimize downtime. thanks for your feedback

Re: Data Migration

Anonymous — Tue, 04 Jul 2017 00:17:05 GMT

1. replicate,
2. start-off another Oracle 11g from the replicated data,
3. then take a dump and import into the final location on Linux
seems to be the best choice but there will be a difference in timestamp on the data,
will i be able to apply achive logs from HPUX Oracle 11g to update the 12c