Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements

Breathe easy -- you now have more time to plan your next steps with Qlik!
QlikView 11.2 Extended Support is now valid through December 31, 2020. Click here for more information.

jblomqvist
Valued Contributor

A question about understanding Symbol tables and Bit-Stuffed tables in how QlikView stores data

Hi all,

I have read this blog post by Henric Cromstrom on Symbol tables and Bit-stuffed pointer tables: Symbol Tables and Bit-Stuffed Pointers

Can anyone please tell me a practical way of identifying how many bit will be required for values?

Values
John
David
Stu
Benny
Christopher
Jankovski

Is there a way of finding how much storage will be required for a table based on Symbol tables and bit-stuffed tables for a table?

3 Replies
MVP
MVP

Re: A question about understanding Symbol tables and Bit-Stuffed tables in how QlikView stores data

You can export the memory statistics from Settings - Document properties - General tab.

Your field shows 6 distinct values, so you would need 3 bits for the pointer.

jblomqvist
Valued Contributor

Re: A question about understanding Symbol tables and Bit-Stuffed tables in how QlikView stores data

Hi Swuehl,

Thank you for this.

So if I had 10 distinct values I would need 5 bits? Does it mean that if I have X number of unique values, if I divide by 2 it gives me the number of bits I need for a pointer?

Also what do those numbers in Ponters mean exactly in the below example?

Symbol+tables.png

It is from Henric's blog above. I am not sure how he got 1101 as Pointer for CompanyName for example. Or 100 for ProductName

MVP
MVP

Re: A question about understanding Symbol tables and Bit-Stuffed tables in how QlikView stores data

I've tried to post some more details, but I am blocked by this awesome forum software to do so.

Binary representation is using power of two. So 3 bits = 8 possible values, one more bit doubles the value range:

4 bits = 16 possible values. And so forth.

So you would need only 4 bits to encode up to 16 distinct values.

I think the images above are kind of misleading when looking at the length of the pointer and the number of visible distinct values. The dots at the bottom should probably denote that many more symbols may follow, but since there are no dots at the top, I wonder why the bit pointer starts with an offset. Maybe it's explained in the blog, I would need to review it. For the amount of values we see, each symbol tables would need 1 bits for the pointers (if zero and one could be used for encoding).