topic Re: Problem with application size in QlikView

Problem with application size

nicolas_martin — Mon, 12 Jan 2015 16:48:26 GMT

Hello,

I have an application. When I save the application without compression, the application is approx 175 MB.

I opened the application with a DocumentAnalyser (thank you Rob), and it says that the Table is 165 MB.

--> ok, the layout and other compotents take about 10 MB, that sounds OK.

Now, I add just 1 new table (no relation with the others), with only 1 line, from an Excel file.

When saved,my application is now 225 MB!

With the DocumentAnalyser, it still says that the Table is 165 MB (few bytes more, because of the new line).

I don't understand why a few bytes increase the size of the application by 50 MB.

Re: Problem with application size

nicolas_martin — Mon, 12 Jan 2015 17:08:43 GMT

If I add at the end of my application:

TMP:
LOAD * INLINE [
foo, bar
];

--> 175 MB (not compressed).

If instead I add at the end of my application:

TMP:
LOAD * INLINE [
foo, bar
1, 2
];

--> 225 MB (not compressed).

Re: Problem with application size

Clever_Anjos — Mon, 12 Jan 2015 17:16:50 GMT

Are foo, bar related to other fields in your app?

Re: Problem with application size

Anonymous — Mon, 12 Jan 2015 17:35:24 GMT

Or, maybe there is another logical table that contains both foo and bar - so the new table creates a synthetic table (?)

Re: Problem with application size

nicolas_martin — Mon, 12 Jan 2015 17:37:53 GMT

No, as I said in my first post, there is no relation with any other table.

Re: Problem with application size

Clever_Anjos — Mon, 12 Jan 2015 17:54:57 GMT

Would you mind posting those info?

1) Table Viewer screenshot

2) Document Properties / Tables

Re: Problem with application size

nicolas_martin — Wed, 14 Jan 2015 11:32:32 GMT

Here is my table viewer:

The table "_LANGUAGE" should contain 4 values "english / french / spanish / german", in order to choose the labels on charts.

If there is no line in the _LANGUAGE, my application "test - language without line.qvw" is 176'560 KB.

_LANGUAGE:
LOAD * INLINE [
_LANGUAGE
];

With the "mem" file, I see it should be 174 597 848 Bytes. --> no many differences

If there is 1 line in the _LANGUAGE, my application "test - language with 1 line.qvw" is 227'334 KB.

_LANGUAGE:
LOAD * INLINE [
_LANGUAGE
english
];

With the "mem" file, I see it should be 174 597 935 Bytes. --> many difference

See attached file "QlikOptimizer.qvw" to see de differences between the 2 files.

I don't explain the gap between my 2 versions of the application.

Re: Problem with application size

marchoctober — Wed, 14 Jan 2015 12:26:18 GMT

Hi!

As you may know there is very helpful post Symbol Tables and Bit-Stuffed Pointers on how Qlik handles the data. Moreover, please export memory usage statistics from first version app (175 mb) and second app (225 mb). You can do that from General tab of properties, button Memory Statistics.

Compare the data values (load stats into Qlik), possibly it could help to understand which field or table is gaining weight with load inline.

Re: Problem with application size

nicolas_martin — Wed, 14 Jan 2015 14:38:23 GMT

please export memory usage statistics from first version app (175 mb) and second app (225 mb)

That's what I have done in my previous post, and I loaded the both "mem" files in the application "QlikOptimizer" and created a chart to compare the two applications.

As I said, my first application (175 MB) have 174 597 848 Bytes of data, whereas my second application (225 MB) have 174 597 935 Bytes of data.

Re: Problem with application size

marchoctober — Wed, 14 Jan 2015 14:49:27 GMT

The question is to compare both of files, are the fileds and sizes are equal. Where are the peeks?

By the way you publish exact amount or amount on hard drive?

Re: Problem with application size

nicolas_martin — Wed, 14 Jan 2015 15:03:00 GMT

The question is to compare both of files, are the fileds and sizes are equal. Where are the peeks?

That's what I've done in the application "QlikOptimizer.qvw", attached in a previous post.

The fields are EXACTLY the sames, except for the field "_LANGUAGE" because there is 1 more line:

			QvwFile	test - language with 1 line			test - language without line
Class	Type	SubType	Id	Bytes	sum(Count)	sum(Size)	Bytes	sum(Count)	sum(Size)
Database	Table	Records	_LANGUAGE	1	1	1,00	0	0	0,00
Database	Field	Symbols	_LANGUAGE	13	1	13,00	0	0	0,00
State Space	Field State	State	_LANGUAGE	1	1	1,00	0	1	0,00
State Space	Field State	Selection	_LANGUAGE	320	0	0,00	256	0	0,00
State Space	Table State	Internal	_LANGUAGE	8	0	0,00	0	0	0,00

so, it should take only few more Bytes, not 50 MB.

By the way you publish exact amount or amount on hard drive?

I don't understand the question.

The file size is the size on hard drive. The size of it's data is what is given in the "mem" files.

Re: Problem with application size

marchoctober — Wed, 14 Jan 2015 15:37:53 GMT

Ok then,

1) Did you try to add Language via txt file for example?

2) And otherwise, did you try to use very different name, f.e instead of Languange Lang, and other values?

3) Try to reduce ALL data from app and make a complete reload.

4) Make new all empty app, copypaste script and reload model, what it will look like wit size?

Re: Problem with application size

nicolas_martin — Wed, 14 Jan 2015 16:46:54 GMT

1) Did you try to add Language via txt file for example?

Yes: txt and Excel.

--> same behavior

2) And otherwise, did you try to use very different name, f.e instead of Languange Lang, and other values?

All the same...

3) Try to reduce ALL data from app and make a complete reload.

Still the same size (225 MB).

4) Make new all empty app, copypaste script and reload model, what it will look like wit size?

!!!

Whith the same script and no layout, my application is 175 MB!

It means that the usage of this value in my labels expressions (approx 50) costs me 50 MB!

It's unbelievable...

Re: Problem with application size

marchoctober — Wed, 14 Jan 2015 16:52:04 GMT

I think its not usage as well, but a kinda of a bug ar similar. Try now copypaste vizualisations, lets see how bigger it will be? I think nothing should happen.

Re: Problem with application size

nicolas_martin — Wed, 14 Jan 2015 17:43:42 GMT

In my application, all my labels are in a formula like

=Only({<index={number_of_price}>} translation)

where "number_of_price" is a key in my Excel file, and all the translations are in different columns loaded as a crosstable .

If the user selects "english" as _LANGUAGE, my label displays the string corresponding.

If I load my application, it's 225 mb;

I replaced all my expressions having this kind of formula by an empty string --> my application is now 175 MB.

There was about 300 occurence of this expression.

So I guess it's the way the values are stored in the application that is weird...

I'll continue to dig...