Quick Tips #14 - Reloads

hps — Wed, 10 Oct 2012 12:12:39 GMT

Quick Tips #14 - Reloads

On MonthName() and DayName() in the load script:

Reloading data in QlikView can be slowed by the use of methods that doesn’t scale well over cores. This can for some functions be avoided with rewriting them using better optimized ones.

Two such examples are MonthName() and DayName(). If they are used in a load script then the entire load will be slower and scale less optimized than it has to be.

Both functions can be replaced by more scalable functions:

Dayname() can be replaced with TimeStamp(Floor(<Your Timestamp>), 'YYYY-MM-DD')
MonthName() can be replaced with date(<Your Timestamp>, 'MMM YYYY')

These two functions are not as time consuming within the user interface of a QlikView application as an expression, due to caching, and does not have to be avoided there as in the load script.

QlikView improvement in 11.20 SR2 (and onwards) no longer requires workarounds as denoted above.

On the rand() function in the load script:

The script generated by pressing CTRL-q twice relies heavily on the rand() function and will generate data on the fly rather than actually loading it from a source. Do not use CTRL-q code or rand() when performing benchmarks of reloads, as rand() will not scale well over cores.

Cheers from the Scalability Team

Re: Quick Tips #14 - Reloads

msteedle — Fri, 21 Dec 2012 13:19:50 GMT

Can you give any more examples of "methods that doesn’t scale well over cores"? Are they documented somewhere? Thanks.

Re: Quick Tips #14 - Reloads

hps — Mon, 07 Jan 2013 08:02:45 GMT

Hi Michael,

The reason for these operations (dayname etc..) to not scale over cores is that for every row the script will query the operating system for date and time.

So any operation that require that sort of information will show a similar behaviour.

This is only true for loading data. Within the QVW deployed on QVS it will scale well.

There is no list available.

Regards,

Hampus

topic Quick Tips #14 - Reloads in Scalability

Quick Tips #14 - Reloads

Re: Quick Tips #14 - Reloads

Re: Quick Tips #14 - Reloads