Well, let say you've decided to go with Talend. Congrats, that's a good choice.
Now, because you probably have some rows to migrate, just take the time to design the best strategy for your case. Will you migrate 1 table after 1 or will you try to paralize some operations?
Also think about rejected records. Will you iterate until no error or will you restart from the beginning in case of error?
The last but not the least, will you directly connect to the source and target db at the same time, or will who export to flat files first, then import to the the target?
Talend is fairly flexible to let you find the best trail for your case. Don't hesitate to try, fail and try again.
And don't hesitate to share and/or ask questions.
You don't us which db. If the source and target engine are the same, it would be simplier to use the export/import functionalities offered by the db. But if you change the engine for example from Oracle to Postgresql, sure you can use Talend for this purpose.
Source database is Oracle 11g R2 Target database is Oracle 12c size of data is around 20TB and I am thinking export will use a lot of system resources, and take a lot of time , i also want to avoid downtime
Well, let say you've decided to go with Talend. Congrats, that's a good choice.
Now, because you probably have some rows to migrate, just take the time to design the best strategy for your case. Will you migrate 1 table after 1 or will you try to paralize some operations?
Also think about rejected records. Will you iterate until no error or will you restart from the beginning in case of error?
The last but not the least, will you directly connect to the source and target db at the same time, or will who export to flat files first, then import to the the target?
Talend is fairly flexible to let you find the best trail for your case. Don't hesitate to try, fail and try again.
And don't hesitate to share and/or ask questions.
20TB data will take a long time. However, I still believe disk block copy is better than doing ETL data integration.
You can either:
- Use block copy replication and turn on replication first into a replica of the Oracle 11g. Then once the db is replicated, you temporarily turn off the replication, take of dump of the replicated db first as a backup, upgrade the Oracle 11g in place to Oracle 12g upgrading all your data files. This will take time but will not impact your production DB too much.
- Use Oracle DB link to link the Oracle 11g from the Oracle 12c. Then do ELT to get the data from 11g to 12c. This will impact your production DB if it is very heavy in processing.
- Use Talend to migrate table by table. Still it will take a long time to do so. This is a good operation if the target DB is in AWS for example. AWS has a service called snowball where you send them your 20TB disks and they load it up for you by literally plugging the disk into their datacenter. You won't upload 20TB. It will take ages unless your connection to internet to AWS is gigabits.
In all the above cases, you will want to be able to replay the log of the Oracle 11g database for the duration of the migration, after the migration is over, so as to bring the migrated db back to the same level as the original one if the original one keeps on running while migration is happening in the background
1. replicate,
2. start-off another Oracle 11g from the replicated data,
3. then take a dump and import into the final location on Linux
seems to be the best choice but there will be a difference in timestamp on the data,
will i be able to apply achive logs from HPUX Oracle 11g to update the 12c
