The Qlik engine - A Forgiving Engine - Qlik Community

hic · ‎2016-03-15

Databases are usually not very forgiving.

Strict rules apply, defining what’s allowed and what’s not. For example, you are not allowed to enter data unless it has the right data type and is formatted the right way. Further, you are often not allowed to enter a value for a foreign key unless this value already exists in the master table. And you are not allowed to enter the same value twice if the field is a primary key.

The reason is of course to ensure data integrity. Without such rules, the database would soon be cluttered with bad quality data and contain a large number of errors.

The fact is that a good system is one that has a large number of rules, but at the same time is easy to use: Equipped with a user interface designed in a way so that the user doesn’t notice the rules – or at least isn’t disturbed by them.

But with the Qlik engine it is a very different situation.

QlikView and Qlik Sense should not make sure that the data is free from errors. Instead, they should do exactly the opposite: Display the source data along with all its errors. This requirement is totally different from the demands you have on a database, and as a result the Qlik engine is built in a different way:

Data Types

There are no data types in the Qlik engine. The reason is simple: You may have data from different tables or even from different data sources in one single field. Then there is a potential risk that you have different data types in the different sources.

When loaded, all fields are converted into duals (number and text, or just text), and so one field can contain data that originally had different types.

Formatting

A single field can have a mixed data format. Also here, the reason is simple: Different sources may have different formats. As a result, it doesn’t matter if a date is formatted as 3/31/16, 2016-03-31 or 42460. They will all three represent March 31, 2016.

Each distinct field value has its own format, and a single field may thus be displayed with different formats.

Referential integrity

The Qlik engine does not enforce referential integrity. For example: You may have a customer ID in your fact table that does not exist in the customer table (which would be an error in the data integrity of the database). But the Qlik engine will accept this and show NULL as customer name.

Relationship type

Often you know if you have a many-to-one or a many-to-many relationship between two entities. But this information is not loaded from the database. Instead the Qlik engine assumes worst case and is always prepared for a many-to-many relationship.

Links between tables don’t carry information about relationship type. And all calculations involve aggregations, since there is a possibility for multiple values of the referenced field.

The bottom line is that the Qlik engine is a very forgiving engine. It handles errors in all of the above cases gracefully. No matter how many such errors you have in the data, the Qlik engine will always make a best-effort attempt in evaluating and showing the loaded data.

HIC