Application Size inflates with new DB

[diwakarnahata]

Hi All,

I am facing a strange issue where i am running the same QlikView script in two Databases (Vectorwise and Redshift), on Vectorwise the qvw file size is 3.5 GB and on Redshift the file size inflates to 8.5 GB.

We are loading 24 months of rolling data in the main fact table, and the # of records for these 24 months are 10% higher in Redshift database. But, that should not inflate the file size by more than twice.

Attached is the export of the .mem file of the two application executed on the Vectorwise and Redshift respectively.

Also, we checked the data types of the 2 main fact tables and 1 relate table which are voluminous, and they are mostly the same between the two databases.

There are serious concerns over the scalability of the application due to this issue.

Can someone suggest, how to debug this issue.

Regards,

Diwakar

[diwakarnahata]

Hi All,

I loaded the data with Compression - None and the qvw file from both the DB were ~12 GB.

I assume that this would be the RAM footprint when application is loaded on Server.

However, when I checked the RAM usage of the Server, with 2 users, the RAM consumption was 110 GB.

Please find below the screenshot from the Management Console.

Please suggest if this is as expected, if not then how to debug this.

Regards,

Diwakar

[jfkinspari]

From the screenshot you can tell, that 3 documents are loaded.

The 110gb of committed memory, may therefore not all be from the application in play.

What you could do was to restart the server service, in order to reset the memory, and then open the application. Make sure that you don't preload any other apps.

[diwakarnahata]

Hi Jens,

Thanks for your inputs. I have removed the unwanted apps now, but still the RAM consumption is high.

The 8 GB application (12 GB with Compression - None) is taking 88 GB of RAM with just 1 user.

Please find below the screenshot.

Are these numbers are normal? Are there any tools available to debug these kind of issues?

Regards,

Diwakar

[jfkinspari]

hmm, try and look at the Open Documents tab to be sure no other documents are loaded into memory

[diwakarnahata]

Yes, tried that as well. Only 1 document is opened.

[jfkinspari]

Now I'm just shooting

1. When the Server service is restarted, is the VMCommitted then close to 0? We want to be sure, that's it the document itself that increases the memory 85 gb when opened

2. Try and enabling preload of the document, to see if VMcommited makes the same as in 1. If not, it might be an (extremely) expensive object, that takes up the memory