Optimised load and UI performance in charts

[sonikajain]

Hi all,

This is basically to understand one of the statements I came across in a qlikview blog.

I know that optimized qvd affects the QVW load/reload time. I read : Optmized QVD will affect UI performance.

So, my question is: Will a chart in a optimized loaded QVW load faster then in a QVW which is non-optimized loaded.

[sunny_talwar]

Thanks Rob!!

Can you explain this a little more?

So the impact of conversion caused by an un-optimized load is an increased memory requirement, which can have a small impact on performance.

What is converting here? Sorry, if this was straight forward.

[marcus_sommer]

Hi Sunny,

here a short calculation-example:

5 years * 365 days = 1,825 days * 8 bytes = 14,600 bytes

5 years * 365 days = 1,825 days * 20 bytes = 36,500 bytes

The difference between both are 21,900 bytes because it happens only against the distinct fieldvalues stored within the symbol-tables (and the data-tables are just connected through a bit-stuffed pointer). Therefore by many applications you won't notice a performance impact between them because the small ones are fast anyway and by the big ones those few bytes more are negligible - either in running/storing-times in script as well as in loading/calculation-times in UI.

If it's not a date else a timestamp with approximately 5,000 distinct times on each day it looked different:

5 years * 365 days = 1,825 days * 5000 timestamps * 8 bytes = 73,000,000 bytes

5 years * 365 days = 1,825 days * 5000 timestamps * 42 bytes = 383,250,000 bytes

Now the difference would be approximately 210 MB and quite noticeable in each way - and this alone from the store of the symbol-tables. But the pointer would also increase and therefore this should be avoided by splitting dates and times in searate fields - both is very well explained here: https://community.qlik.com/blogs/qlikviewdesignblog/2014/03/11/the-importance-of-being-distinct

This meant to focus on performance-aspects by the big fields/tables and loadings will ensure that the overall performance is sufficient and all the small improvements could be made if there is a bit of unoccupied time (what usually is seldom). I assume this is what Rob had meant …

- Marcus

[sunny_talwar]

Marcus -

Thanks for going over this with an example. It all makes sense except one important thing... how did you come up with these numbers 8 bytes vs 20 bytes when dates and 8 bytes vs 42 bytes when it is timestamp? What determines the size (bytes) here is sort of I am unable to understand completely.

Once again thank you for going over the example, it definitely help me understand the impact....

Best,

Sunny

[marcus_sommer]

Hi Sunny,

it's from my playing here: ByteSizes from Values in QlikView and was here just an assumed size and could in dependence with the choosen format differ - in lesser bytes using a "standard" time-format and even more by using formattings which include the full day- and monthsnames:

- Marcus

[sunny_talwar]

Now this makes a lot more sense.... so, I understand two things from this

1) Date/TimeStamp dual take more space then Date/TimeStamp string? Is this right?

2) Because Rob mentioned this

So the impact of conversion caused by an un-optimized load is an increased memory requirement, which can have a small impact on performance.

it seems that an optimized load will load the Date/TimeStamp as string because it takes less space (better performance) vs. an un-optimized load which will load it as Date/TimeStamp dual. But isn't that a problem that an optimized load will load Date/TimeStamp as string? Or am I missing something here.

Thanks for your continued help Marcus, now that we are talking about this, I am really curious to know what is going on.

Best,

Sunny

[marcus_sommer]

The first is right - a timestamp-string would need less space as a dual-timestamp because the dual-value needs to store the numeric part too and probably also a flag to mark the value as dual.

An optimized load like an unoptimized load will keep a date/timestamp (it's either already directly stored or flagged within the meta-data) as it was. Only the unoptimized load applied a formatting on field-value level if the field is flagged as date/timestamp and has a formatting - for this take a look on my screenshot above for <Type> and <Fmt>.

- Marcus

[rwunderlich]

See also

http://qlikviewcookbook.com/2017/03/dual-storage-vs-dual-behavior/

-Rob

http://masterssummit.com

http://qlikviewcookbook.com