idea Re: Qlik Replicate - Google Bigquery replication with the source table partition keys in Suggest an Idea

Qlik Replicate - Google Bigquery replication with the source table partition keys

skokanay — Tue, 08 Mar 2022 08:45:50 GMT

Currently, Qlik Replicate task is unable to replicate the source table partition key (ex : Teradata) while creating and loading a data into GCP Big query dataset tables. Ideally as per principles or fundamentals of data replication the same behaviour from source i.e including the DDL with its respective partition should be taken care during the replication process.

The current behaviour is Qlik replicate only create the table with flat structure of source in the target GCP Big query without including the partitions.

Hence request to consider this idea/enhancement request for your future releases. as you know there is an huge demand to migrate the on-prem data into GCP cloud in particular Big query for analytics and having this feature implemented asap is an good idea.

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys

john_wang — Tue, 08 Mar 2022 08:48:17 GMT

Hello @skokanay ,

Thank you. it's relevant to support case #00025433: Qlikreplicate: GBQ table is not creating with partitions

Regards,

John.

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys

skokanay — Tue, 08 Mar 2022 09:40:30 GMT

Yes John, We had to use the work around steps. As discussed with QLIK CSE we have posted this in the ideation so that its taken care in coming release.

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys

bobvecchione — Thu, 17 Mar 2022 17:52:03 GMT

This is a great request and it has been added to our priorities for GBQ improvements. I do not have a timeline for availability yet but will provide one as soon as I can.

--bobv--

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys - Status changed to: Open - On Roadmap

Nulee_Massaro — Tue, 22 Nov 2022 13:50:14 GMT

This is on the roadmap.

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys

Nulee_Massaro — Thu, 02 Feb 2023 14:34:45 GMT

Thank you for your request - a few questions:

Is it related to the table creation, i.e. create the table with the partitioning details?
Do you need to support DDLs related to partitions like add/remove/more partition?
What are the sources that you are working with?

Let us know. Thank you!

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys

jcjames — Fri, 17 Mar 2023 01:30:32 GMT

Hi @Nulee_Massaro ,

See below the responses from the customer.

1. Is it related to the table creation, i.e. creating the table with the partitioning details?

Yes correct, this is for creating the tables with the partitioning

2. Do you need to support DDLs related to partitions like add/remove/more partitions?

Yes, the same properties from source db need to be replicated / inherited to the target db (for example : its source TD to target GBQ)

3. What are the sources that you are working with?

Teradata, MS SQL, Oracle and PostGres .

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 16:00:16 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Qlik Replicate - Google Bigquery replication with the source table partition keys - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 16:00:18 GMT