Group by Single threaded, why?

Tue, 09 May 2017 13:06:45 GMT

Hi,

Anyone knows why Group By inside qlikview Script is single threaded (only uses one cpu core).?

Am I doing something wrong ( ex server settings, NUMA?) or is this the default behavior? Is there a way to overcome it (workaround or something)

Thanks.

PS. Logically I can do group by in a large dataset by splitting the dataset to x parts (1 per available core) group by each part , and then group by the results.

Re: Group by Single threaded, why?

tresB — Tue, 09 May 2017 13:19:15 GMT

This is how hic‌ describes it:

...The script evaluation is inherently different from the analysis evaluation of data. In the script, the order of the records matter and this poses limitiations on the algorithm. We have made the script execution multi-threaded, but in principle the multi threading is on a record per record base: A thread needs to wait for the other threads to finish before it can process the next record.

Further, both the crosstable transformation and the group by transformation involve multiple records, which also makes multi threading difficult (or maybe impossible).

The discussion thread is here : Single-threaded calculations

I can't claim that I fully understand this though. I thought at least part of it could have been gone parallel in execution.

topic Re: Group by Single threaded, why? in QlikView

Group by Single threaded, why?

Re: Group by Single threaded, why?