Qlik Community

New to QlikView

Discussion board where members can get started with QlikView.

Announcements
QlikWorld 2020: Join us May 11 - 14, 2020 in Phoenix, AZ. Register early and save $400. Learn More
qlikview979
Valued Contributor

Data Compression technique

Hi experts,

What is data compression technique?

2 Replies
arulsettu
Honored Contributor III

Re: Data Compression technique

storing table into qvd

ex;

store tablename into c:\folder\tablename.qvd;

MVP
MVP

Re: Data Compression technique

QlikView and Qlik Sense almost always creates a symbol table with a list of the unique values in a field together with pointers to the actual values and where they are. This is a technique called a symbol table but in general its a de-duplication technique too and leads to a data compression on two ways. First the complete list of non-unique values will be much compressed as long as there is a ratio of unique to non-unique values that is higher than 1 plus the overhead of maintaing the symbol table with pointers. Furthermore the pointers are bit-stuffed which leads to even better compression.

All this is done automatically for you while the load script is running. The running applicaton is making use of it without needing to worry about it by the developer of the App or the Analyst/User.

This is employed in QVDs and for the in-memory associative databases. In fact it is one of the major parts of it. Indexing is also taken care of on the fly by this same technique.