Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello,
I just have a couple general questions about the full load process and its relationship with server disk/memory and how exactly the captured information is stored then loaded.
I know that during CDC, the changes are cached in memory and if there are too many then it get stored on disk.
However I don't know much about how memory/disk works with Full Load.
These are my questions:
Is the data from full load stored on the Replicate Server before being loaded onto the target?
If so then how do massive tables get replicated?
If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?
Thanks,
Mohammed
Hello Mohammed @MoeyE ,
Thanks for reaching out!
Is the data from full load stored on the Replicate Server before being loaded onto the target?
If so then how do massive tables get replicated?
Nope.
The flow goes through memory. Replicate unload/read records from source DB (by a chunk, eg a given number of rows, default 10000 rows, see sample below) and then load/write to target side DB, repeats the operation until all rows 'copied' from source to target.
If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?
In current versions the Full Load stage does not support broken/resume, the table/s (or whole task) need to be reload if something wrong prior to full load completion. The reload can be at table level (see below sample) or task level (for all tables) by task --> Run --> Reload target...
Feel free to let me know if you need any additional information.
Regards,
John.
Hi John,
Thank you, that pretty much summed up what I wanted to know
Regards,
Mohammed
Hello Mohammed @MoeyE ,
Thanks for reaching out!
Is the data from full load stored on the Replicate Server before being loaded onto the target?
If so then how do massive tables get replicated?
Nope.
The flow goes through memory. Replicate unload/read records from source DB (by a chunk, eg a given number of rows, default 10000 rows, see sample below) and then load/write to target side DB, repeats the operation until all rows 'copied' from source to target.
If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?
In current versions the Full Load stage does not support broken/resume, the table/s (or whole task) need to be reload if something wrong prior to full load completion. The reload can be at table level (see below sample) or task level (for all tables) by task --> Run --> Reload target...
Feel free to let me know if you need any additional information.
Regards,
John.
Hi John,
Thank you, that pretty much summed up what I wanted to know
Regards,
Mohammed
Glad to hear that Mohammed!