Suggest an Idea

[tdphill1064]

Instead of doing loads in Snowflake during Batch process where the Snowflake table is zeroed out to have no records.  Create a staging table to pump the data in then do the switch of the main table and staging table to limit the downtime of the loading process.  It allows for users to access the old data until the new data is added.  

[Jesse_W]

This would be very useful for larger tables that we are loading.  It would be a good option on the loads to use a staging table or not.

[bobvecchione]

Hi Folks,

Thanks for this request. I can share that we are making a number of Snowflake target improvements with Replicate and will consider this as a part of that list.

Once we know more around timing/dates...I will share here.

--bobv--

[tdphill1064]

 Thanks for the reply.  It will be excellent if they can get this added.  

[Nulee_Massaro]

Thank you for reaching out. Can you please elaborate on the issue you are having with Snowflake? What is your main concern? 

[tdphill1064]

The main concern is that when you do a batch load Qlik zeros out the table requested when doing a full load.  I have seen other products use a method of having a stage table and letting the main table still contain all the data.  Once the full load is done in the staging table the replication software switches the staging table to the main table naming convention and instead of having 30 minutes to hours for the load to complete there is an outage only on the switching of the table names which happens in milliseconds.  

[AlexOmetis]

Yes, this would be really useful to limit the impact of long-running full loads - either when pushing DDL changes that haven't been auto-processed, as part of recovering from an issue or where change data capture isn't available. 

[Meghann_MacDonald]

From now on, please track this idea from the Ideation portal. 

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.