topic Re: Scaling Rserve Deployments? in Integration, Extension & APIs

Scaling Rserve Deployments?

Sat, 16 Nov 2024 09:31:35 GMT

The Qlik R extension relies on the RServe server. There are multiple solutions to scale Rserve deployments, such as HAproxy and Nginx, for Qlik installations. When done correctly, this contains benefits such as better response time, better uptime and load balancing. Nginx looks to be still beta for Windows Server so HAproxy looks to be the most appropriate solution, currently. In the past, HAproxy seemed to provide better solution in load balancing. I would like to hear your experience.

Scaling Rserve Deployments:

1. What are the preferred ways to scale Rserve deployments particularly in crowded multiuser Qlik installations?

2. Are there plans for Qlik to support load balancing in RServe or is this something for which community should provide solutions for?

3. Are there some guidelines, recommendations or estimates about when load balancing could be an issue in the RServe deployments?

Re: Scaling Rserve Deployments?

Steven_Pressland — Fri, 21 Jul 2017 10:05:47 GMT

some good feedback. It is worth noting that Rserve is NOT recommended for use on Windows, so especially any scaling scenarios will need to run on Linux for the reasons outlined by Rserve (not a Qlik issue).

Also note that the projects are open source so you can adapt the code to my your specific needs and feed that back to the project. Perhaps a scaling feature would be a good addition.

Re: Scaling Rserve Deployments?

Sun, 23 Jul 2017 19:49:36 GMT

Thank you for your answer. Unfortunately, I cannot find a mention about the platforms in Rserve documentation.

1. To which are you referring with "outlined by Rserve"? This document [1]?

2. Can you outline the reasons you are referring with "reasons outlined by Rserve"?

[1] https://cran.r-project.org/web/packages/Rserve/Rserve.pdf

Re: Scaling Rserve Deployments?

Anonymous — Mon, 31 Jul 2017 09:20:31 GMT

Hi Henri,

There is no clear statement about whether to recommendations for Rserve on Windows, but the maintainer of the Rserve package has made a number of remarks that he doesn't use Windows. (here: Rserve stil crashes on Windows with invalid escapes sequences · Issue #22 · s-u/Rserve · GitHub)

"Sorry, no, I don't use Windows. Also note that this is not a crash - it's merely R exiting on an error. Since the main purpose of Rserve is to provide parallel sessions while Windows is incapable of that, it is a very rare exception to use Windows and Rserve. Windows support (if present) is mostly provided as a contribution from the community or friendly folks at Revolution Analytics. I will certainly entertain pull requests to restore Windows functionality and to fix any Windows-specific issues."

Also here are a few limitations on the Rserve for Windows version: Rserve for Windows - release notes

"The Windows version of Rserve is slightly limited compared to its unix counterpart. Quick overview of the differences: (for technical details see bottom of this page)

the config file is Rserv.cfg in the working directory (unless changed at compile-time)
initialization of R_HOME is performed automatically by fetching the information from the registry, i.e. you should use R installed by the official Windows installer and leave the DCOM option checked (this is the default) unless you know what you're doing. You should copy Rserve.exein the bin directory of such installed R.
no parallel connections are supported, subsequent connections share the same namespace
sessions are not supported - this is a consequence of the fact that parallel connections are not supported
no error handling - always use "try(...)" when evaluating commands, otherwise the server may die on an error
local unix sockets are not supported (by design, as the name implies)"

Especially the parallel connections limitation can impact your capability of building a scalable environment.

Regards,

Bas