Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
MoeyE
Partner - Creator II
Partner - Creator II

Does Full Load store anything on the Replicate server?

Hello,

I just have  a couple general questions about the full load process and its relationship with server disk/memory and how exactly the captured information is stored then loaded.

I know that during CDC, the changes are cached in memory and if there are too many then it get stored on disk.

However I don't know much about how memory/disk works with Full Load.

These are my questions:

Is the data from full load stored on the Replicate Server before being loaded onto the target?

If so then how do massive tables get replicated? 

If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?

Thanks,

Mohammed

 

 

Labels (1)
2 Solutions

Accepted Solutions
john_wang
Support
Support

Hello Mohammed @MoeyE ,

Thanks for reaching out!

Is the data from full load stored on the Replicate Server before being loaded onto the target?

If so then how do massive tables get replicated? 


Nope.

The flow goes through memory. Replicate unload/read records from source DB (by a chunk, eg a given number of rows, default 10000 rows, see sample below) and then load/write to target side DB, repeats the operation until all rows 'copied' from source to target.

john_wang_0-1678929365714.png

 


If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?


In current versions the Full Load stage does not support broken/resume, the table/s (or whole task) need to be reload if something wrong prior to full load completion. The reload can be at table level (see below sample) or task level (for all tables) by task --> Run --> Reload target...

john_wang_1-1678929523902.png

 

Feel free to let me know if you need any additional information.

Regards,

John.

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!

View solution in original post

MoeyE
Partner - Creator II
Partner - Creator II
Author

Hi John,

Thank you, that pretty much summed up what I wanted to know

Regards,

Mohammed

View solution in original post

3 Replies
john_wang
Support
Support

Hello Mohammed @MoeyE ,

Thanks for reaching out!

Is the data from full load stored on the Replicate Server before being loaded onto the target?

If so then how do massive tables get replicated? 


Nope.

The flow goes through memory. Replicate unload/read records from source DB (by a chunk, eg a given number of rows, default 10000 rows, see sample below) and then load/write to target side DB, repeats the operation until all rows 'copied' from source to target.

john_wang_0-1678929365714.png

 


If the source database crashes during full load will full load complete or will it need to wait for the source to become available again to continue?


In current versions the Full Load stage does not support broken/resume, the table/s (or whole task) need to be reload if something wrong prior to full load completion. The reload can be at table level (see below sample) or task level (for all tables) by task --> Run --> Reload target...

john_wang_1-1678929523902.png

 

Feel free to let me know if you need any additional information.

Regards,

John.

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!
MoeyE
Partner - Creator II
Partner - Creator II
Author

Hi John,

Thank you, that pretty much summed up what I wanted to know

Regards,

Mohammed

john_wang
Support
Support

Glad to hear that Mohammed!

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!