Performance: Single big node or better to have multi-node set-up where resources are split?

[mwallman]

Hello,

Our NPrinting usage is growing and currently we have a single node environment where we have 16 core CPU and 200GB RAM server.

We are wanting to create a better environment and wondering what would be better. Make the single node bigger or try and go for a multi-node to manage the load?

In multi-node I was thinking:

- What if we could set-up so that we have 3 nodes each server with 10 CPU cores and 100GB of RAM each. So in total 30 cores and 300GB of RAM.

Is that better than a single node with 30 cores and 300GB RAM?

My thinking is a multi-node can give you better resiliance compared to a single node but on the down-side it's more servers to manage.

I appreciate the answer can "depends" but I am looking for some guidance. Generally the NPrinting server config is OK but we are trying to make it better.

What are your thoughts? What would you favour and why? Any other pros and cons to each approach could also be good as part of the consideration.

[Lech_Miszkiewicz]

First question I normally ask is the license. How many engines does your license cover? Typically it is 1 engine only so in such scenario there is no point talking about multinode.

On the other hand talking purely about benefits I think having more engines would be beneficial as dealing with other processes in multinode environments can be handled better (PDF printing for example). 

The last question is why would you need 100 GB of ram? That would be pure waste of resources. Does your NPrinting really need that much ram as it will never use it? Does it get anywhere close to the cap at the moment? People often think NPrinting consumes huge amount of resources, but it is not true - all hard work is done in Qlik Sense/QlikView and NPrinting is only producing reports.

as always pointing to documentation - notice that large NPrinting deployment is the 3 engines (20 CPU and 32 GB ram). The  

https://help.qlik.com/en-US/nprinting/September2020/Content/NPrinting/DeployingQVNprinting/Multi-eng...

[mwallman]

Hi Lech,

Your post made me read this Help page on Performance: https://help.qlik.com/en-US/nprinting/September2020/Content/NPrinting/DeployingQVNprinting/Performan...

It seems to imply that CPU cores is more important than RAM i.e. the more processors, the better the NPrinting performance. RAM is not as much important.

I guess what I want to know is, it seems that it's the account supplied in the Connection that opens the Qlik Sense app in the Qlik Sense server, interacts with it.

So where exactly does the report building happen, in the NPrinting server?

Let's say you were adhering to the best practices for reports but still experience performance issues, a better option would be to expand the number of CPU cores?

In our environment we only have a license for one engine server but I wonder whether it's better to ask for more engines and when expanding CPU cores, do that by spreading the work in multiple nodes? Would that be better compared to increasing more CPU cores in a single node?

[Lech_Miszkiewicz]

90% of performance is sitting in the Qlik Sense performance.. you can have "super-hyper" NPrinting server but without well working Qlik Sense app you are in no mans land. So focus on Qlik Sense performance is the main thing to do... and worrying about NPrinting performance is the last thing to worry about.

[mwallman]

Sure, I still would like to know what the NPrinting server does because the Performance page in Help suggests growing the CPU cores but it's not quite clear how the NPrinting server uses it's resources given that RAM is not used that much.

Again this is how I am reading the Performance help page so just wanting some clarification and understanding.

Open to NPrinting support staff commenting here too.

[Lech_Miszkiewicz]

This is how much I understand. On single server configuration when you run Server and Engine on 1 box CPU is used to run server and cpu's are used to open independent sessions to Qlik Sense by using web renderer in -built browser. 1 CPU core is required to best of my knowledge to run such session. The more cores you have the more parallel sessions you can run. From Qlik Sense point of view 1 CPU is equivalent of a 1 concurrent user using Qlik Sense. Now what happens on NPrinting then.....

What NPrinting server does:

• extracts data from Qlik Sense UI tables (equivalent of export data when you click on the chart)
• extracts images from charts
• pastes data/images into templates (copy/paste)
• creates files (xls, doc, ppt, html etc) or prints them to PDF (yes PDF from XLS is actually print of the excel file)
• saves produced reports
• sends reports (pushes them to NewsStand, Hub or SMTP server)
• runs algorithms to perform scheduling and task queues 
• etc 

So all those tasks are simple and dont require much of RAM or CPU. Now when you think about it the amount of RAM required for NPrinting reports production is the amount of RAM needed to open, preview etc the largest report x-times, where X is equal to number of CPU. This calculation is not documented nor recommended by Qlik, but always worked for me.

So the assumption would be:

• I need 4 CPU and 8 GB Ram - that is my starting point usually
• I want to add another 10 Cores so i can get more parallel reports running - given that my Qlik Sense/QlikView is capable of dealing with such demand. That gives me MAX 14 parallel sessions running (14 CPU for engine. btw: this is already more than recommended if you configure connection to QlikView) 
• If my largest report is 120 MB i would need 14x120 = 1.7 GB extra RAM
• That means that I should be totally fine by adding just another 8 GB of ram to my server (that gives me in theory a lot of breathing space)
• Final config would be then 14 CPU and 16 GB RAM

Now with such configuration I can observe what happens on the server when we run reports. Do not get confused by the fact that you would like to trigger 200 reports at the same time, that all of them will run at the same time. NPrinting server will put most of them in the queue and will start production based on available resources. You can monitor behaviour of the server, see CPU and RAM consumption and make adjustments if needed.

Ram and CPU may also depend not only on number of reports produced, but number of users visiting News Stand. At the end NewsStand site is hosted on NPrinitng server and its usage may consume server hardware. 

[mgranillo]

@Lech_Miszkiewicz  Your post is very helpful but I'm still confused about some documentation. From Nprinting help page it says: "The amount of RAM required for Qlik NPrinting Engine computers depends on the QlikView documents you use to create reports. On a Qlik NPrinting Engine, each instance of QV.exe opens a single QlikView document. For example, if your Qlik NPrinting Engine server has four CPU cores it means that it runs four QV.exe instances which in turn open four QlikView documents. You need enough RAM to adequately manage these four QlikView documents to ensure good performance."

Does this apply to Qlik Sense as well?  I have a 16 core engine so does that mean my QVF will be in RAM 16 times. That would take a lot of RAM: 200MB on disk expanded to maybe 2 GB in RAM then x 16 cores = 32 GB.  Is it opening on the Qlik Sense server or the nprinting server? 

[Lech_Miszkiewicz]

Hi @mgranillo 

No it does not apply to Qlik Sense, in fact it does not apply to all QlikView connections neither.

It is more complex. I will break it down below as it all depends on connections type:

• Qlikview
  • Connection type: LOCAL = qvw file can be opened locally on nprinting machine as many times as many CPU cores you have (like using qlikview desktop File--> Open option). Amount of RAM needs to be sufficient to do that. If your app in RAM has 1GB and you have 4 cores you need at least 4GB +(margin for operations of OS and operations and calculations on those qvws) which easily can be another 4GB
  • Connection type: QVP or Claster = qvw files can be opened on QlikView Server as many times as many CPUs you have (like using qlikview desktop File--> Open in server option). That moves all heavy loading to QlikView Server and you need to ensure that QlikView server is able to handle cocurrent sessions 
• Qlik Sense
  • NPrinting can open Qlik Sense apps concurrent sessions on Qlik Sense server as many times as many CPUs you have (just like QlikView QVP or Cluster). In your case since you have 16 cores it actually can run 16 concurrent (very active Qlik Sense sessions), so your Qlik Sense server must be powerfull enough to handle it! NPrinting server will not require that much ram. 

Hope that clarifies it.

cheers