A question about understanding Symbol tables and Bit-Stuffed tables in how QlikView stores data

[jblomqvist]

Hi all,

I have read this blog post by Henric Cromstrom on Symbol tables and Bit-stuffed pointer tables: Symbol Tables and Bit-Stuffed Pointers

Can anyone please tell me a practical way of identifying how many bit will be required for values?

Values
John
David
Stu
Benny
Christopher
Jankovski

Is there a way of finding how much storage will be required for a table based on Symbol tables and bit-stuffed tables for a table?

[swuehl]

You can export the memory statistics from Settings - Document properties - General tab.

Your field shows 6 distinct values, so you would need 3 bits for the pointer.

[jblomqvist]

Hi Swuehl,

Thank you for this.

So if I had 10 distinct values I would need 5 bits? Does it mean that if I have X number of unique values, if I divide by 2 it gives me the number of bits I need for a pointer?

Also what do those numbers in Ponters mean exactly in the below example?

It is from Henric's blog above. I am not sure how he got 1101 as Pointer for CompanyName for example. Or 100 for ProductName

[swuehl]

I've tried to post some more details, but I am blocked by this awesome forum software to do so.

Binary representation is using power of two. So 3 bits = 8 possible values, one more bit doubles the value range:

4 bits = 16 possible values. And so forth.

So you would need only 4 bits to encode up to 16 distinct values.

I think the images above are kind of misleading when looking at the length of the pointer and the number of visible distinct values. The dots at the bottom should probably denote that many more symbols may follow, but since there are no dots at the top, I wonder why the bit pointer starts with an offset. Maybe it's explained in the blog, I would need to review it. For the amount of values we see, each symbol tables would need 1 bits for the pointers (if zero and one could be used for encoding).