Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Can we run Pre SQl or Post SQL statements for a table load in Qlik replicate
We are planning to leverage qlik for all data ingestion needs, currently we cannot use replicate for tables where we dont have primary keys
We have few cases where we do extract last few months of data everytime and then delete that same months from target table and re insert them , why cant we do this in qlik replicate
We sometimes just do a full cdc on tables in middleware server, compare source table with target table and then insert new rows and delete changed rows from target , using hashing of rows
it would be great to have pre sql , post sql support at table level & custom query as source features in replicate
This will eliminate the need to have use ETL tools for data ingestion
Hi @choudarybabu ,
We have the Data Movement gateway cloud version to cover all the replicate and ELT use cases in one place. Please reach out to your account representative\sales team for more information.
Thanks,
Swathi
--but my question is mostly around a post/pre SQL to be executed on destination db before the load
Unfortunately No. Replicate cannot but this feature is available in Compose.
Purpose of Replicate (as the name suggests) is to get your data from source to target. We can add supported transformations but that's the overall scope. You can check if any of the available options for Global or table level transformations would help with your requirement.
Thank you,
Hi @choudarybabu ,
We have the Data Movement gateway cloud version to cover all the replicate and ELT use cases in one place. Please reach out to your account representative\sales team for more information.
Thanks,
Swathi
Trying to understand your requirement and my in-line answers for each of the mentioned pain point:
We are planning to leverage qlik for all data ingestion needs, currently we cannot use replicate for tables where we dont have primary keys
-- Please elaborate, mentioning the source and target DB that's in scope. There can be a workaround for most of the cases.
We have few cases where we do extract last few months of data everytime and then delete that same months from target table and re insert them , why cant we do this in qlik replicate
-- We can do this in Replicate Full Load process by adding pass thru filters and defining the date conditions.
We sometimes just do a full cdc on tables in middleware server, compare source table with target table and then insert new rows and delete changed rows from target , using hashing of rows
-- Replicate Full Load with "Do Nothing" mode would help get only the changed records between the source and target.
Please find more details in the Replicate user guide on each of the options suggested.
https://help.qlik.com/en-US/replicate/May2022/Content/Replicate/Main/Introduction/Home.htm
As @SwathiPulagam mentioned the new Cloud "Data Movement gateway" is in the pipeline but it's concentrated more on Cloud Databases and we need to understand your specific requirement.
Thank you,
hello Shashi,
thanks for the reply, I apologize for the late response .
Source is Oracle and destination is Snowflake
We have used pass thru filters, thats helpful to an extent
My requirement here but can we use custom queries as source (may be i can workaround this by creating a view with custom sql)
but my question is mostly around a post/pre SQL to be executed on destination db before the load
can we implement pre/post sql on target db with Qlik replicate
--but my question is mostly around a post/pre SQL to be executed on destination db before the load
Unfortunately No. Replicate cannot but this feature is available in Compose.
Purpose of Replicate (as the name suggests) is to get your data from source to target. We can add supported transformations but that's the overall scope. You can check if any of the available options for Global or table level transformations would help with your requirement.
Thank you,