topic Re: Memory increases when formating a timestamp? in QlikView

Memory increases when formating a timestamp?

Mon, 02 Jun 2014 21:08:47 GMT

Hi.

We are formatting a timestamp field by the use of TimeStamp()-function - and for some reason this increases the size of the resulting QVD file.

Why is that?

Info:
1. This is QV 11
2. When reading the timestamp field without formatting through the TimeStamp()-function the resulting QVD file takes up 40 MB, but using the TimeStamp()-function increases the resulting QVD file to 60 MB.
3. The number of distinct values is the same in both loads.
4. The field is in both cases a dual, i.e. a timestamp format with an underlying number
5. The .mem statistic for the qvw file doesn't show any difference or answers to the difference

Any explanation for the 50 % increase?

Re: Memory increases when formating a timestamp?

jolivares — Mon, 02 Jun 2014 21:37:11 GMT

Review this document...

http://community.qlik.com/blogs/qlikviewdesignblog/2014/03/11/the-importance-of-being-distinct

Re: Memory increases when formating a timestamp?

rbecher — Mon, 02 Jun 2014 21:42:44 GMT

HI,

the reason is that a formatted timestamp will take additional memory for the string representation in QVD file and RAM. So I suggest to stay with the default format (or change it to what format you need) and not to use TimeStamp() function.

Btw. this has nothing to do with the blog link in the answer above which talks about memory saving by splitting timestamps into date and time field.

- Ralf

Re: Memory increases when formating a timestamp?

Mon, 02 Jun 2014 21:56:38 GMT

Hi RB.

Thanks.

I guess you are referring to the dual format left after using the TimeStamp() function.

Could you explain why it is that the first load (without formating), which is also representeted by a string (when put into a listbox) with an underlying number, is not a dual consuming the same number of MBs in the resulting qvd?

Re: Memory increases when formating a timestamp?

rbecher — Mon, 02 Jun 2014 22:27:15 GMT

Because it is not stored as dual in the QVD. It's dual on the UI level only, I guess.. You should see the difference if you take a binary look into the QVD file. If you find a readable timestamp (the text representation) it will consume more memory.

- Ralf

Re: Memory increases when formating a timestamp?

rbecher — Mon, 02 Jun 2014 22:41:05 GMT

Btw. I was a bit wrong in the first place. You should use TimeStamp() function but without format parameter..

Re: Memory increases when formating a timestamp?

Mon, 02 Jun 2014 22:42:56 GMT

Hi again.

I will have a look into the qvd file. A little strange it is though.

Re: Memory increases when formating a timestamp?

Mon, 02 Jun 2014 22:44:33 GMT

That I did - but with the MB consuming result.

Re: Memory increases when formating a timestamp?

rbecher — Mon, 02 Jun 2014 22:45:35 GMT

It's not strange but a kind of optimization..

Re: Memory increases when formating a timestamp?

rbecher — Mon, 02 Jun 2014 22:54:42 GMT

Then my first guess was right.. 😉

But in this case you need to format the field in all UI objects which could be an effort.