topic Re: Documenting Lineage between a SQL Server and Snowflake in Catalog and Lineage

Documenting Lineage between a SQL Server and Snowflake

jonATVSP — Tue, 11 Mar 2025 20:51:23 GMT

Our OLTP is in SQLServer (source) and our DW is in Snowflake (target)

The SQL Server is replicated via Qlik Replicate and the EDW is then loaded Via Qlik Compose. So we have

SQLServer -> Qlik Replicate -> Qlik Compose -> Snowflake

My question is "Will Qlik Catalog document the lineage from the SQL Server and Snowflake?

Re: Documenting Lineage between a SQL Server and Snowflake

Christopher_Ortega — Tue, 11 Mar 2025 21:12:25 GMT

Unfortunately, based on the setup that you've described, Qlik Catalog will not capture all of this lineage.

Replicate lineage is provided to Qlik Catalog via Enterprise Manager but Compose lineage is not. As such, you will see SQL*Server to Snowflake landing lineage through the Replicate job. However, if you catalog your Compose Snowflake targets in Catalog there will be no lineage connecting them to the preceding lineage.

Re: Documenting Lineage between a SQL Server and Snowflake

jonATVSP — Tue, 11 Mar 2025 21:21:25 GMT

I do not think that we use Compose to catalog the Snowflake targets. If this is the case, then could we see lineage from Snowflake back to the replicate piece? I think that this is what I understood by your initial note.
Thanks! Jon

Re: Documenting Lineage between a SQL Server and Snowflake

Christopher_Ortega — Tue, 11 Mar 2025 21:38:25 GMT

There isn't a catalog'ing capability within Compose. I was simply referring to the fact that you could catalog those tables manually in the Catalog. Their lineage however would not be known to Qlik Catalog.

Re: Documenting Lineage between a SQL Server and Snowflake

gonelbo1 — Sat, 12 Apr 2025 13:14:11 GMT

If you only need lineage to the table and data source level, there are tons of data governance and lineage tools on the market. Some of the tools you are using might already have some of this capability. Would need to know more about your tools to dig into that possibility.