When you mention numbers of 'cores' do you mean numbers of virtual processors ?
For the physical host how many physical cores does it have ?
The reason why I ask the above is that for a virtual server its number of virtual processors should always be less than half the number of physical host's, physical cores. Otherwise processing can end thrashing with context switching and performance plummets.
I had a scenario [non QlikView] some years ago with a physical host with 4 cores, that had a virtual server with 2 virtual processors that was performing badly. Somebody decided to double the virtual processors to 4 and its performance become even worse.
I was called in and against much opposition decreased the virtual server processors to just 1 virtual processor and performance improved dramatically. I did win the bet and was made a lovely cup of tea.
So I would say if your physical host has 32 or less physical cores then your findings above are exactly what I would expect.