R Integration with Qlik Sense

Sonja_Bauernfeind · Feb 23, 2021 4:26:23 AM

How to get started with R-Script integration within Qlik Sense

The architecture of the integration at a High-level looks like this:

Qlik Sense Advanced Analytics integration is essentially an extension to Qlik Sense’s expression syntax, and as such it can be used in both Chart Expressions, and in Load Script Expressions.

With this new capability, we are now able to add syntax to a chart expression that tells Qlik Sense that particular expression should not be evaluated on the Qlik Sense server, but instead, all the information and data needed to calculate that expression should be sent via the server side extension on to the backend R system for calculation.

After the advanced analytic calculations are completed, the data is sent back to the Qlik Sense Server and to the client for visualization.

Environment:

Qlik Sense Enterprise on Windows , all versions

Resolution:

This video shows an example of how Qlik Sense connects to an R server for extending expression capabilities in Qlik Sense Apps while offloading the calculations to the R server engine.

Click here for Video Transcript

In order to start displaying a simple "Hello World" in Qlik Sense using a R-Script, we will do the following:

1. Have R & R-studio installed in your system. (RGui included with R for Windows can also be used) R can be downloaded at https://cloud.r-project.org/

2. We need a package in R to extend R functionality to applications via TCP/IP. The package name is "Rserve()"
Install the package using the below command in RStudio GUI:

install.packages('Rserve')

3. Now we need to invoke that library and start Rserve. In order to do so, execute the below scripts:

library(Rserve)
Rserve()

4.The communication method from Sense to R is taken care using gRPC. R is not a supported language in gRPC by default.

So a possible solution for this is to develop a connector in any of the supported languages of gRPC. Qlik provides an open-source connector developed in C# which in turn access Rserve to be able to run R scripts.
qlik-oss/sse-r-plugin
Once you built the connector, start the SSEtoRserve.exe (ideally on the Rserve server itself)

Note: Qlik Support does not support this plugin directly. Inquiries should be submitted via GitHub under sse-r-plugin - Issues

5. Now we will have to configure the plugin:

For Qlik Sense Desktop June 2019 and up:

Add the following line in the settings.ini file:

SSEPlugin=R,localhost:50051

The settings.ini is located in this location:

C:\Users\[userid]\AppData\Local\Programs\Qlik\Sense\Engine\settings.ini

For previous Qlik Sense Desktop:

Add the following line in the settings.ini file:

SSEPlugin=R,localhost:50051

The settings.ini file is located in this location:

C:\Users\(username)\Documents\Qlik\Sense

For Qlik Sense Enterprise/Server:

a. In the QMC, add a new Analytic Connection.
b. Restart the Qlik Sense Engine service.

Please refer to the screenshot below for creating a new connection.

Note: If the R-Plugin (SSEtoRserve.exe) was installed on the R-Server (where Rserve runs) or another machine, point to that machine name instead of 'localhost'. Also, in multi-node environments with multiple Qlik Sense Engines, even if the plugin was installed on the Central node, make sure to add the Central node's hostname instead of 'localhost' as the other Rim node Engine services need the correct DNS/Netbios name to reach the plugin.

6. Now Open a Qlik Sense App and add a KPI object in the sheet. This can be one of the Apps included with the plugin itself under <storage path>\sse-r-plugin-master\sense_apps

Note that the example apps also need data connections to be created to the data files included with these apps files in the above location.

7. Otherwise, a new app can be created and any data may be loaded for the SSE example below.

8. For the measure, add the following expression which contains an R-script:

R.ScriptEvalStr('paste(q$firstWord, q$secondWord);', 'Hello' as firstWord, 'World' as secondWord)

9. If everything is configured properly, the R-script shown in bold above should be executed fine and it should display a "Hello World" message.

Understanding the R-script & Function:

R.ScriptEvalStr('paste(q$firstWord, q$secondWord);', Only([First Word]) as firstWord, Only([Second Word]) as secondWord)

Eight script functions are automatically added to the functionality of the plugin. What is needed to be covered on the plugin side to fulfill the functionality is to implement the EvaluateScript rpc function.

The syntax of these functions is:

<EngineSSEName>.<FunctionName>(Script [,Parameter...])

Where the Script is an R-Script to be evaluated & Parameter is the data sent from Qlik's end.

Here, we use the ScriptEvalStr function which accepts argument of type String & returns a String. The 'paste' function in R concatenates vectors after converting to character. We pass two data fields of type string from Qlik (First Word & Second Word). The R-script then references these data fields through the q dataframe (structure already taken care in R) (q$firstWord and q$secondWord). The script/function finally returns a String back to Qlik Sense.

Related Content:

s-puddle · ‎2021-01-25

Hi @Sonja_Bauernfeind

Hoping you can help. We previously had R installed on our server when we were on a single-node. We are now on multi-node and have installed R on a separate server from our Qlik Sense servers. What settings do I need to change to for the analytics connection?

Thanks

Andre_Sostizzo · ‎2021-02-03

@s-puddle , The Analytic Connection for R should point to the SSEtoRServe plug-in, via port 50051. This can be installed anywhere including on the RServe server. For example: If the plug-in was installed on the Qlik Sense server, and the RServer on a remote server, the following parameter needs to be changed on the file below.

File:

C:\<install path>\sse-r-plugin-master\csharp\SSEtoRserve\bin\Debug\SSEtoRserve.exe.config

Parameter:

...

<setting name="rserveHost" serializeAs="String">
<value>add the hostname of Rserve server here</value>

...

I will update the article soon with information around this.

Thank you for your question!

KoenD92 · ‎2021-02-04

Hello @Sonja_Bauernfeind, I've set up a connection between QlikSense and R, using your guidelines. Everything is working perfectly, however we have noticed that both RServe and SSEtoRserve use up large amounts of RAM. When starting up both services, they use less than 20 MB of RAM combined. After running my R script on a large dataset, Rserve goes up to about 350 MB, and SSEtoRserve to 800mb. This does not lower after the script is done running. Do you know of any way to flush this memory?

Sonja_Bauernfeind · ‎2021-02-04

@KoenD92 Let me see what I can find for you!

I'll take this to our subject matter experts, though this may not be a Sense specific topic. In the meantime, you can also ask this question in our forums to see if someone in the community can assist you directly.

Sebastian_Linser · ‎2021-02-04

Hello,

as a suggestion can you check if you can

add the DISCONNECT statememnt in Qlik script at the end of the script:
https://help.qlik.com/en-US/sense/November2020/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptRegu...
on the R side I would look for the same

On the other hand it looks like you already reached out in the right spot for your question:

https://github.com/qlik-oss/sse-r-plugin/issues/57

since the plugin you use is open source, not supported by Qlik directly.

julian_rodriguez

Hello Experts,

What could I be doing wrong?

R Integration with Qlik Sense