Is there any explanation of the .mem file?

[#1]

Hi!

I have been using DocumentAnalyzer created by Rob Wunderlich and it is truely an awesome application.

Since the app is using the mem file I have always wondered what is stored in the mem file and how the numbers are calculated.

The question has became more relevant this week since a customer of ours asked how eg. Size was calculated and what the number meant.

I could not give a satsifying answer and none of the developers in my group could either.

Googling and searching this community did not really give anything so now I post the question on the community.

Can anyone give a good explanation of how the mem file is created and what the data in the file means.

Kind Regards,

Andreas

[rwunderlich]

The mem file is generated by pressing the Mem button in Document Properties General tab, or via the API call SaveMemoryStatistics (as DocumentAnalyzer does).

As for understanding the .mem file, loading it into the QlikviewOptimizer app (should be a copy around here somewhere) can help you to make sense of it. I don't know of any one public document that explains the QV storage layout and the mem file. I do believe it's covered in the Advanced Topics course of V11.

You'll find some information on field sizes here:

http://qlikviewnotes.blogspot.com/2008/05/when-less-data-means-more-ram.html

http://qlikviewnotes.blogspot.com/2008/05/memory-sizes-for-data-types.html

The "Symbol" (subtype=Symbol in mem file) space in qlikview contains the set of distinct value of a field. The "Size" of the field (Symbol) is the avglength of each entry. DocumentAnalyzer calculates # of bytes for a field as (# of symbols * symbol size).

A "Record" (row - subtype="Records" in mem file) in a Qlikview table is made up of field pointers that point to an entry in the symbol table. The Size of a Record is the width of pointers for all the fields in that row.

When DocumentAnalyzer calculates the Referenced/Unreferenced bytes chart on the Memory sheet, it considers both the Symbol space and the Record space for each field.

I'm glad you're enjoying the tool.

-Rob

http://robwunderlich.com