Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I've been using the Document Analyzer to clean up my dashboard. On the recommendation tab, it lists every numeric field that I pull into QlikView (pulled in from CSV created from SAS program) and states the field occupies more than the optimal # of bytes.
Is there a way to set all numeric fields I pulled in to a numeric format at once to occupy the optimal 8 bytes? Also, is this something that designers/developers typically do? Looks like would save me potentially .16 GB
There aren't such settings or other automatics which could adjust your numeric fields. The best way is to adjust those fields already within the (very first) load-scripts (on generator level by extracting the data from sources and storing into qvd's).
- Marcus
Thanks for the response, Marcus! I am currently pulling in the CSVs into QVDs (I run through 3 tiers) using a LOAD *... are you suggesting that each numeric field be converted manually in a T1 layer (converting CSV to QVD)?
To do any transforming within the T1 layer is against the pure doctrine but sometimes it's more practically to make some small adjustments like those formattings.
For example if your source-data contained fields from a calculation like gross price / sales tax = net price then you could get numbers with a lot of decimal places but you didn't need them because you need only an on the cent rounded value. In this case you could directly save processing and storing bytes per value and you will decrease the amount of unique values for this column - here is an example of timestamps but the logic is the same: The Importance Of Being Distinct and with them the needed size of the qvd will be smaller.
But it will be depend on your amount and kind of data and how they will be loaded and which development- and maintaining-strategies you applied where the best place for transformings is. If there is no critical bottleneck on the performance, time-windows for updates or other important reasons I would try to make my development- and maintaining-measures as consistent as possible.
- Marcus
I would "break the rule" and make them numeric in the T1 layer if possible.
-Rob