Qlik data lineage in a hybrid environment

[sebastiannord]

Hi,

We have QVD:s that are transformed in Qlik Sense Enterprise on Windows  and transferred to Qlik Cloud with Qlik DataTransfer where they are read into the end user app.

Is it possible to view whole data flow ie QS on Windows -> Qlik Cloud? Currently we can only see them in separate views.

Example, data flow:

1. App A extracts data (QS on Windows).
2. App B transforms data on (QS on Windows).
3. QVD:s created by app B are transferred to Qlik Cloud using DataTransfer.
4. App C reads the data created by app B (Qlik Cloud).

We wish to see the complete flow from app A to C when using data lineage in app C.

[Christopher_Ortega · Accepted solution]

Based on your question I assume that you are using the Qlik Lineage Connectors to see the lineage for the on-premise application.

We don't currently have the lineage from Qlik Data Transfer but it is in our plans to accomplish what you have requested.

[sebastiannord]

Thank you for your reply @Christopher_Ortega.

That is correct, we are using the Qlik Lineage Connectors to see the lineage for the on-premise application.

Then we just have to wait for this feature to be available in the future so we can se the complete data flow.

[callanml31]

Dbt-synapse adapter works great. Not sure if you tried it. Also not sure what you mean when you say it doesn't work well with spark? The dbt-spark adapter seemed to work well for me, especially on databricks, which is unneeded now since they made their own adapter in dbt-databricks.

[hermis]

If you're just learning the bare minimum about dbt necessary to make it run complete replacements on your star schema, its probably worse than sql in SSIS because you won't have really learned anything yet that makes dbt useful. In my opinion you really need to be taking advantage of features like automating the merges, templating with Jina to reduce the code you type, different targets for dev and prod, tests, pre and post hooks, etc for it to be worth learning and using. Once you get ther it is, but its not superior from day 1