topic Re: growing data size with RAND() in QlikView

growing data size with RAND()

Tue, 23 Jul 2013 07:01:29 GMT

Hi.

I was trying to reload Data from my qvw-file(150Mb) with Binary for making a new Demo.qvw.

I was decided to recalculate Data values with *Rand(), but after reloading my new Demo.qvw have +130Mb size.

Numbers of rows(24 000 000) and tables the same. Structure the same. Only fields value differ.

I tested on the other files, results the same.

Why?

BINARY [Data.qvw];

table_1_demo:

NoConcatenate

LOAD

Country,

Dept,

Value*Rand() as Value

Resident table_1;

DROP table table_1;

Re: growing data size with RAND()

Gysbert_Wassenaar — Tue, 23 Jul 2013 07:32:40 GMT

Rand() returns random values. Sometimes a value will occur more often than other times. That means the number of unique values will be different every time you reload. The number of unique values in a field determines how well that field can be compressed. A good randomizer will create nearly only unique values, so compression will be very low. See this blog post: Symbol Tables and Bit-Stuffed Pointers

Also, chart objects and background images are not loaded with a binary load. That may explain more of the size difference.

Re: growing data size with RAND()

Tue, 23 Jul 2013 07:58:22 GMT

Thanks.