As a general rule, provided the development of the applications in the server is optimal, there is no need to free up any memory. QlikView is designed to work with as much memory as possible, without conflicting with the OS. That means that there are a lot of cached expressions and therefore the user experience will be better.
If you restart, you are making your users to start caching again so they are not benefitting by QlikView technology.
Check the Working Set limits in the QMC, System, Setup, QlikView Servers, Performance to make sure that the max limit leaves room enough for the operating system and other backup and antivirus software running.
Thanks for your answer. We know it will impact the user behavior for Qlikview server and we have limited memory and we will upgrade our hardware in months. But before that we need to restart our service or do you have any suggestion on that?