QVS has its own memory manager. QVS will collect and keep every bit of memory that it ever had a need for. The VM size for this process will only grow, except if you restart the service. Only then, memory will be released. Don't worry, this is a good thing.
A QVS size that is often near the absolute max. is indicative of RAM under pressure. Imagine that a big reload job tries to start? it won't get any memory from QVS, and there won't be any spare left in the system.
Ashfaq's spreadsheet can help you in determining the optimal amount of RAM. Note you should add a safe amount for reloads if this platform also runs the QlikView Distribution Service. Add the memory size of the largest reload job (single task chain), or of the largest set of reload jobs that run in parallel.