Read directly from the source database (without Replicate)

[michelabarsottini]

Hi all,

I have a question regarding Compose: can you read data directly from another database? We have another replication software which has a staging area in a database outside the Replicate-Compose environment and we would like to connect directly to this database and include the data in the tasks. Is it possible or we have to use Replicate to replicate the data from this database to the Landing Zone?

Thank you,

Michela

[shashi_holla]

@michelabarsottini 

The Full Load on Compose might still work but not the CDC. I think better to have replicate read the data from the other database and load it to Landing.

Thank you,

[Clever_Anjos]

Maybe custom ETL´s can do what you want to achieve - https://help.qlik.com/en-US/compose/May2022/Content/ComposeDWDL/Main/DW/Data%20Warehouse/create_cust... 

This might be useful if you have your project consuming data that is being landed by Replicate and a few tables that are outside of the scope of Replicate

[TimGarrod · Accepted solution]

Qlik Compose does not move data across the network.  It is built as a ELT based solution, leveraging Replicate in order to move data from a disparate source database to the data warehouse landing zone. 

Compose only connects directly to the data warehouse target and processes used a set-based ELT methodology.   It can process any data that resides in your data warehouse platform into its data model with the automated ETL.    So, the question is - is your 'other database' where you have staged data within your datawarehouse environment? 

e.g if its SQL Server or Snowflake - is it just sitting in another database that is accessible from the Compose connection (select * from OTHER_DB.SCHEMA.TABLE).   If so - you can assign that database as a landing connection and Compose can process the data. 

If the database is on a different technology / instance (eg a different SQL Server instance / server) - then no, Compose does not provide the ability to move data across the network. 

Hope this helps.

[michelabarsottini]

Hi All,

thank you for all your responses, we wanted to read directly from another instance, but if it is not possible with Compose we will use Replicate instead.

Thank you for your time.

Michela