Single and Multi-threading in QlikView and Qlik Sense

[Sonja_Bauernfeind]

All functions in QlikView and Qlik Sense are multi-threaded. There is a wide-spread misconception in the field that some functions are single-threaded and others are not. For some additional insight see A Myth About Count(distinct...)

But there are some operations that are single-threaded. It is best described below:

Fig 1: Multithreading

The first phase – Logical inference – is multithreaded. It just calculates which field values are possible and which are excluded.
 
The last phase – Calculate aggregation – is multithreaded. It performs the actual calculation, e.g. Sum(Sales). Then a long table is split into several chunks and each thread takes care of one of the chunks.
 
The middle phase - Find values or combinations - is however, single-threaded. Or rather, it has one thread per object on the screen. This means that you can have a pivot table that is single-threaded. The single-threaded phase figures out which combinations there are to consider in the aggregation. If you have Product as a dimension, and Sum(Sales) as a measure, this phase finds the combinations between Sales and Product. “Which Product does this sales transaction belong to?” So, this phase builds look-up tables so that a specific Sales record can be attributed to a specific Product. Normally, this is a fast operation, so you don’t even notice that it happens.
 

There are cases where this phase can be very CPU-consuming:

• If you have more than one very large fact table
  Say that you have four tables: Customer – Order Headers – Order Details – Products, and the two middle ones are large (millions of records). Then it can be expensive to have both Customers and Products in the same calculation, since the Qlik engine on-the-fly needs to generate all combinations between the two large tables. Better then to join the two tables already in the script.

• If you have fields from more than one source table inside your aggregation
  Say that you want to calculate Sum(ListPrice*Quantity), and ListPrice and Quantity are in two different tables (Products and Order Details). Then the Qlik engine needs to generate a temporary table with all combinations between the two, in addition to attributing each record in this temporary table to the dimensional values. Better then to move ListPrice to the Order Details table already in the script.

Related Content:

• QIX Engine Memory Management and CPU Utilization 
• Understanding Qlik Sense Performance 
• QlikView and Qlik Sense resource usage and the Qlik Associative Engine Memory Management and CPU Uti...