Prevent implied use of dual...worth the trouble?

[markp201]

So we have our data and some fields (e.g.dates) have strict typing.  They are either numeric or string.

I've read that using num and text will override and reduce the document size.

Dual is convenient but most of the time, references to a date or string field are within the type.

Overriding dual would require the use of interpretation functions (e.g. num#)

In general would it be better to explicitly set the type and manage in the front-end or use dual.

Right now the app is over 1GB so we're mulling over the use of dual.

[Gysbert_Wassenaar · Accepted solution]

Try analyzing the document first with the Document Analyzer tool created by rwunderlich‌. It's very well possible that you'll find other things that have a larger impact on the size of your document.

• Document Analyzer  Qlikview Cookbook‌
• The Document Analyzer Compare Tool | Qlikview Cookbook

[rwunderlich]

There are two kinds of duals (in my mind). The first kind is one you may create yourself like:

Dual('Yes',1)

This kind of dual stores both the string and numeric value. The symbol size of this field is 11 bytes. 3 bytes for the "Yes" and 8 bytes to store the numeric 1 (numeric values always use 8 bytes).

A second kind of dual is what I call "dual behavior".  An example of this is a proper Date(). Today's date number is 42802. The symbol size is 8 bytes, only the numeric is stored.  The format pattern eg 'MM/DD/YYYY' is  an attribute of the entire field, not the specific field value. When the string representation is needed, the format pattern is applied on demand. The string value is not stored with the data.

What I'm calling a "proper date" is already optimized for storage. Sometimes, dates get converted to the "full dual" -- string and number together during loads. The format in the Document Properties, Number pane will show as "mixed".  I see this frequently with unoptimized QVD loads.

Document Analyzer will attempt to identify these "unoptimized dates" in the recommendations section, along with other numeric fields that may have the same issue.

In my experience, you don't get much memory back from this issue.  But, as Gysbert suggested, Document Analyzer may help you to find other savings and tuning opportunities.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[markp201]

Based on other postings on limiting distinct values, we broke up the datetime fields that we break up into parts.  Right now, we're at 100M+ and response time has been reduced about 80%.  Question about DA - are there plans to use it with QVFs?  For now, we can put our script into a qvw because we're primarily interested in optimizing the data model.   We do have some overly complex expressions and the plan is to move as many as possible to the model.  Thanks for the replies.