Had the same issue, and here is a simple way to automatically kill QV / QVS / QVB if using too much memory. QVS can be restarted by adding one more line "net start QVS" , wich does nothing if the process is running fine (meaning not not killed due to excessive memory )
Recommend to set the threshhold around 95% of phisical memory for dedicated QVS servers. Of course, you need to study trends to find the valid memory limits for reload tasks (qvb.exe) and developer tool (qv.exe) in your environment .
Old qlikview posts are gone, so here is a link to Blogger : http://alexandrutoth.blogspot.com/2010/03/qlikview-memory-limits.html