idea Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property in Suggest an Idea

Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property

agoergen — Thu, 30 Sep 2021 17:19:16 GMT

In Snowflake, there are built-in features to backup updated/deleted data called "time travel" and "failsafe" and while these are useful for your organization's data, for Qlik we do not need to keep history for the Control tables such as "attrep_status" and "attrep_txn_state". We can control this by creating the schema where these control tables are placed with an attribute of "Transient" and "DATA_RETENTION_IN_DAYS = 0" but this is a manual intervention. As Qlik Replicate usually creates these schemas on our behalf, we would like this to be an option when connecting to Snowflake as a target. This way we're not being charged for storage on updated/deleted rows in these Qlik control tables.

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property - Status changed to: Open - New

Shelley_Brennan — Tue, 19 Oct 2021 13:44:22 GMT

Replicate does provide an option to specify a specific schema for the control tables: https://help.qlik.com/en-US/replicate/May2021/Content/Global_Common/Content/SharedEMReplicate/Customize%20Tasks/control_tables.htm

Would this suffice for your use case?

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property

agoergen — Tue, 19 Oct 2021 13:48:54 GMT

It gives us a workaround. It allows us to create the schema ahead of time, with the properties we want. However this is a manual task per schema, and we replicate into lots of schemas.

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property - Status changed to: Open - Collecting Feedback

Shelley_Brennan — Tue, 19 Oct 2021 14:32:50 GMT

Understood. Thanks for the quick reply. We will consider this as we do our future planning and collect feedback from others here as well. Thank you!

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property

Prabodh — Wed, 03 Nov 2021 20:07:42 GMT

We are also replicating data to a lot of schemas to SF. However, we only maintain one copy of audit tables in one schema.

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property

agoergen — Wed, 03 Nov 2021 20:24:18 GMT

In my experience, Qlik creates a control schema in every target it replicates to (every database in Snowflake). If you have it set up to maintain all of your control tables in one place, can you share where this is configured? We have about 20-30 Qlik Replicate tasks, each targeting a different Snowflake database.

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property

Prabodh — Wed, 03 Nov 2021 22:03:28 GMT

Ah! We are only replicating to 1 Snowflake database, but multiple schemas in that database.

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 16:30:14 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Snowflake: Qlik Replicate should create it's Control Tables in Snowflake using Transient schema property - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 16:30:16 GMT