I would strongly recommend to deal this directly with Qlik Support.
As a first step, increase the Logging Level to "Debug Logging" for the QDS, run manually the task or chain of tasks which causes the error and verify, you will see which process has each task running, and which user started it.
(2017-12-11 17:32:36) Information: Opened the QlikView Engine successfully. ProcessID=2364
If you have several CONNECT statements or you are using NPrinting, this could be one cause.
I have not experienced this behaviour in either bare metal or VMWare VMs or AWS instances. We sometimes also run out of engines, but because we have a lot of tasks running at the same time.