Understanding ETL in Qlik Sense - Need Clarification
Hello everyone,
I have a question regarding the ETL process in Qlik Sense, which I'm struggling to understand. I would appreciate any help in explaining it better or correcting me if I'm wrong.
I have extensive experience with Power BI but am currently working at a company that exclusively uses Qlik Sense. I've noticed several differences between the two platforms, but what is confusing me the most is the scripting. For each dashboard, there are three scripts for Extraction, Transformation, and Loading.
From what I understand, the extraction script is used to retrieve data from the database (in this case, Oracle) and save it into a QVD file. The transformation part is unclear to me, and the loading script seems quite similar to the transformation script. This is where variables are created from the data in the QVD files and where you can proceed to the folder and build the dashboard.
Does anyone use this approach? I haven't found much information about this process online.
Another aspect that confuses me is whether it is possible to use QVDs from other dashboards for extraction. If so, is this considered good practice?
Any insights or explanations would be greatly appreciated. I’m working hard to understand everything, and a comparison to Power BI would be extremely helpful if someone has that knowledge.
The transformation stage involves cleaning, filtering, and preparing the data for analysis i.e. joining tables, applying business logic, or aggregating data. While it might seem similar to loading, the key distinction is that transformation shapes the data into a usable format, often saving the results in a separate QVD.
The loading stage, on the other hand, involves bringing the transformed data into the application to build visualizations, where you can create variables, apply additional filters i.e. show only the sales figures for the current quarter, or perform final data manipulations i.e. calculating a new field, such as profit margin, by subtracting costs from sales before presenting it to the end users.
Reusing QVDs across different dashboards is not only possible but also recommended for efficiency. This practice avoids redundant data extraction processes, making data loads faster and more efficient.
However, it’s important to ensure that the data in these QVDs is relevant and up-to-date for the dashboards that rely on them.
The transformation stage involves cleaning, filtering, and preparing the data for analysis i.e. joining tables, applying business logic, or aggregating data. While it might seem similar to loading, the key distinction is that transformation shapes the data into a usable format, often saving the results in a separate QVD.
The loading stage, on the other hand, involves bringing the transformed data into the application to build visualizations, where you can create variables, apply additional filters i.e. show only the sales figures for the current quarter, or perform final data manipulations i.e. calculating a new field, such as profit margin, by subtracting costs from sales before presenting it to the end users.
Reusing QVDs across different dashboards is not only possible but also recommended for efficiency. This practice avoids redundant data extraction processes, making data loads faster and more efficient.
However, it’s important to ensure that the data in these QVDs is relevant and up-to-date for the dashboards that rely on them.
