QlikView Expressor: Using Partitions in an Input Operator

QlikView Expressor supports the feature of partitions, which are multiple threads of execution within an operator.  Each thread carries out the same processing on a subset of the records.  That is, (with the exception of broadcast partitioning) each record is processed by only one thread but all records are processed.

 

One of the valuable uses of partitioning is to improve the speed at which records enter the dataflow and the SQL Query and Read Table operators support partitioning by creating multiple connections to the source database.  The SELECT statement exeucted on each connection includes a different WHERE clause, which means that only those records satisfying the WHERE clause are returned on that connection.  By using multiple partitions, network and database delays may be lessened.

 

When your dataflow uses the Read Table operator, as a step in configuring the operator you select a column that contains an integer value  on which to base partitioning.  Expressor then applies a mod function to that value in each record, generating an integer that is less than the number of partitions.  This integer identifies the partition on which the record will be processed (partition numbers are zero based).

partition.png

 

When your dataflow uses the SQL Query operator, you include a partitioning clause in the SQL SELECT statement you encapsulate within the schema used by this operator.  Within this clause, you again specify a column that contains an integer value.  However, with the SQL Query operator, the partitioning clause may be more complex where the partitioning directive may be contained in an AND or HAVING clause rather than within the WHERE clause.  Note that with the HAVING clause, the partitioning directive refers to the aggregate function.

partition2.png

partition5.png

partition8.png

 

Regardless of which operator you choose, you must also specify the number of partitions as a separate operator property as shown for the Read Table operator in the first figure in this document and for the SQL Query operator in the following figure.

partition_count_setting.png

The following video shows you how to configure input operator partitioning.