Qlik might be using the RAM for itself towards the cache. Based on the working limits (lower) set up qlik starts freeing up the cached memory once it’s reached lower limit. Ideally there should not be any issue. If you want to free the memory you can restart the server.
QVS will use as much RAM as necessary and won't release it up to the working set limits as Ramya indicated. The default value for the Low Working set is 70%. You can confirm this in the QMC > System > Setup > QlikView Server > QVS@ > Performance tab. And, as Marcus stated, this is working as designed.