Semantic Layers

[#1]

how to building Semantic Layers in Qlikview

[Anonymous]

Semantic Layers are more of Business Objects thing and the concept is not really a valid one in QiikView.

[petter]

Thank you for asking a very interesting question - I meet this question often, but not as often as I would like to... It is very much the essence of BI - not the term - not the "vendor product" part of it - but the value you deliver to your customers. Whether that is internal or external customers.

A semantic layer is not just a vendors marketing term or property - although Business Objects (now SAP) has successfully - according to Wikipedia - "developed and patented the concept".

It is all about making sense out of your data in every way. And that is exactly what QlikView and Qlik Sense does in a multitude of ways. Qlik hasn't made a big point out of it and separated it out as some kind of module or product where you build a "semantic layer". Qlik has embedded different ways of creating your semantic layer all the way through your development and design process.

This list gives you a hint on some of the tools you use to create the semantic layer:

• Connecting to your data sources and relieving your analysts and users the burden of knowing WHERE the data is coming from.
• Transforming the data by merging tables and removing columns.
• Adding calculations to the data which eventually leads to a new and more semantically appropriate data model. The new data model will be much easier for your analysts and users to relate to - but they still have to use their business knowledge and skills to make business decisions based on the data and the ease of use that QlikView and Qlik Sense naturally offers - but also how you leverage that into an App that makes sense to them.
• Enriching the technical terms often found in many sources - by using for example RENAME FIELDS USING <mapping-table> to rename from technical source terms into semantic/business field names, RENAME TABLES USING <mapping-table>. COMMENT FIELDS USING <mapping-table>, TAG FIELDS USING <mapping-table> ...... so it is actually your effort in employing these Load Script statements to enrich and create a semantic layer. Unfortunately it is not possible to point to a single part of QlikView/Sense or a QlikView/Sense model and say here is your semantic layer. A good semantic layer is the sum or rather more than the sum of a lot of things you do in your "development/design" process. But rest assured QlikView and Qlik Sense do provide you with many ways of creating your semantic layer. So much that you don't really need to promote it as a semantic layer because it makes perfect sense.
• Creating data lineage not automatically but by carefully and skillfully using TAGS and COMMENTS. Maybe even making separate QlikView applications visualizing the semantic layer. The Qlik Data Governance Dashboard is a tool helping you to harvest some of the meta data which documents your semantic layer - meta data always enrich the semantics although meta data is only one important component.
• Employing the Dual() function to create both textual representations of your field values directly connected to the numeric values 1:1 in one single field - which goes unfortunately a bit unnoticed on a lot of QlikView developers.

I do see the perspective where it is useful to build products like  Cognos Framework Manager and Business Objects (some part of it) to force your developers and BI-organization onto the straight and narrow path. That is not the philosophy of Qlik. So you will have to employ more of your skills to achieve this - but QlikView and Qlik Sense doesn't leave you out in the cold.

So my view on this is that a semantic layer is something more of the soul of the solution you have made when you have finished of your development of a QlikView / Qlik Sense application. It is nothing that you can pull out and point to and say "here is your semantic layer". But if you work skillfully all the way through the development process it is not very hard to document it and in the end you will be able to point to what comprise your practical approach to the semantic layer and meta data.

Hope this clarified the subject somewhat - I got a bit carried away. I think Bill Markham's perspective is a valid one to. As long as your customers/users find the application very useful and appreciate it - who cares about the semantic layer? The proof is in the pudding.

[#4]

Hi im new in qlik and it caught my attention as a candidate tool in organization. You already mentioned that in qlik there no need for a semantic layer , so my questions are:

- Can you use multiple heterogeneous  data sources and combine them? For example: been able to explore traditional databases(multiple types) , excel and test files , hadoop hdfs ,hadoop impala and hive without having to integrate them in a common storage repository by an ETL or data integration step ?

- If there is no semantic layer, how does qlik provides final non technical users access to the data? Is there any way to create an abstraction of the data structures so business users can explore an consume the data using business terms instead of technical database field names or querying languages ?

- If there a way to create this abstraction, could you give it security so not every user can access every field, or row in the data? (row and column security)

- If this abstraction exists  , could it be published on the platform  as a central data analysis point? I mean, all new reports/dashboards or tools use this object as a source and all of them would have the same common behavior , and if some day business rules and logic changes, you just change this abstraction and all reports or tools consuming it would reflect the changes without having to change every existing report?

This are all features found on semantic layers on existing tools(for example universes in business objects) , but a feature we are interested in and would be nice if qlik provides is:

- If theres this abstraction to the data, could it be edited,updated and enhanced by non technical business users ? And then all users exploring or reporting based on this abstraction use the business rules defined ? Our use case would be something like:

- An initial data abstraction is published by IT to business users.

- With time business rules and logic changes so for example if a new metric its defined by the business, a business users takes the initial abstraction and with no technical language creates  the new metric based on existing ones,for example: A= B+C

-This is published to user community so can be consumed

- And new reports/dashboards and tools are created using this new metric.

- The business realizes that the calculation has to be A= B*C instead of A=B+C so the business user modifies the data abstraction, publish again and all existing products reflect the change without having to go and change queries in every existing product or new products are created that use this metric and the update is reflected as soon as the abstraction is published.

And if this can be done a desirable feature but no necessary is : can a log be preserved where reports can be generated of who modified the abstraction?

I now its a very large question but it would be valuable to us.

Thank you!