There are a number of things to consider in how Qlikview uses memory. You're correct in saying the .QVW is stored on disk and then QV upon openning it, loads the entire data model into RAM. As a very rough guide, the size on disk is usually about 8 times smaller than in RAM.
If you are using the QV Server, then this will occur on the server's RAM. If a second user opens the same app, then it will only add a small(ish) more to the RAM, the third, even less, etc.
Do you have a more specific concern you need to address? For example, your server is struggling and your RAM consumption is really high?
Do you have access to the server? A couple of things to check:
1. ping time to the server (just in case it's the network)
2. on the server, check the CPU and Memory usage.
I did have a case once where our server (which had 96GB of RAM) nearly ran out of RAM, which was ridiculous for the size of files in use. By restarting the QVS service on the server, it brought the RAM back down.