Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Feb 8, 2023 6:18:53 AM
Apr 2, 2013 5:16:51 PM
If you have never thought about integrating predictive analytics and QlikView, may this article pique your interest on the topic. If you have thought about the potential, but do not know how to get started, may this be the motivation to start your journey.
This demo example illustrates the art of the possible around the interoperability of the R statistics engine and QlikView 11 for enabling predictive algorithms within QlikView today.
R (http://www.r-project.org/) is a free software environment for statistical computing and graphics, which supports predictive modeling and scoring. The example also introduces Rattle (Rattle: A Graphical User Interface for Data Mining using R -- http://rattle.togaware.com/). Rattle was used to create the predictive model and generate the R code implemented within the example QlikView application. The use of Rattle is not detailed here. It is left up to you to explore.
The example assumes you understand R and how to create predictive models in R and how scoring works in R. If you understand how to do those things in R and you know how QlikView works, this example should give you an idea of how:
-- all without the end user having to leave QlikView.
As in the previous example posted by Elif Tutuk, "Integrating QV with R example kit.zip" (http://community.qlik.com/docs/DOC-2975), the example utilizes statconnDCOM. A slightly different approach in this example overcomes a break in functionality introduced into that earlier example by the changing in the behavior of one of the COM API methods in the most recent releases of statconnDCOM.
The integration is done using QlikView's built-in VBScript support to interoperate with the DCOM-based API. This is just one way to integrate with R. With a little research you can discover other potential opportunities for R integration (e.g., OpenCPU). Also, note that R is just one such example of a statistical engine being integrated with QlikView. Other solutions, such as commercially available statistical engines or predictive scoring engines may also be integrated in similar ways.
To get started with this example, save the attached file, QlikView-R_Predictive_Demo.zip, to your system. Open the zip file, and copy the enclosed folder (QlikView-R_Predictive_Demo) to a location of your choice. The location is not important as the example is portable and has no dependency on being placed in a particular folder on your system. Now navigate into the folder you just extracted and find the document named “Qlikview-R Integration Demo 20170310.pdf”. Open the document and follow the instructions step-by-step.
UPDATE (May 7, 2014): I updated the installation document (Qlikview-R Integration Demo 20140421.pdf) to eliminate the RAndFriends installer as it has been unavailable for some time. This edition of the document references R 3.1.0. Hopefully the instructions will continue to be a good guide for subsequent versions as well. I will continue to monitor for changes in R, statconnDCOM and other R components which could effect how the demo application works going forward. The example QlikView application itself remains unchanged. It is a Personal Edition version, so you should be able to explore it with a licensed QlikView Desktop or the Personal Edition. Thank you for your continued feedback on this demo.
UPDATE (September 21, 2015): Everyone who has been seeing the message "The license that is embedded in this document is not valid": I attached an updated version of QlikView-R_Predictive_Demo.zip. Ensure you overwrite the old QVW with the new one in this package. That is the only change. Please forgive this looooong overdue update.
UPDATE (October 9, 2016): Updated the embedded license to work with QlikView Desktop Personal Edition.
UPDATE (February 2, 2017): Updated the embedded license to work with QlikView Desktop Personal Edition.
UPDATE (March 10. 2017): A dependency for this particular example is statconnDCOM from the team at autstat (http://www.autstat.com/). If you wish to utilize statconnDCOM, you will need to contact autstat directly and purchase a license or request an evaluation license for testing.
Hello Piotr,
Thank you for downloading the demo. Kendra, back in April, conveyed similar behavior. Recently I was able to see the behavior firsthand. I believe I understand what is going on. The underlying cause, in my case, was not,in actuality, that the staconnDCOM API was not interoperating with R. Rather, it was a matter with the encoding of the data file in my particular demo. If the statconnDCOM API is working properly in your environment, what I will share with you may fix your issue. BTW I learned you can test the statconnDCOM API with a utility which comes installed in the statconn program folder. That utility should eliminate the API to R connectivity as an issue.
Here is the solution which worked in my case:
1) Delete the QlikView-R folder hierarchy you extracted from the demo zip file.
2) Redeploy the entire QlikView-R application from the original downloaded zip archive.
3) Open QlikView Desktop and go to Settings -> User Preferences.
4) Select the Export tab.
5) Under Default Export Options, choose ANSI encoding. I believe UTF-8 is the default.
6) Now try clicking the Run Risk Assessment button. If all goes as expected, the statconnDCOM banner should pop up and go away (without the macro editor window opening). The Risk of Mortality Decision Tree plot should also change and display the current date/time.
7) If that works, select a different set of Patient (IDs) and click the Run Risk Assessment button again. If your data changes and the plot updates, you’re in business.
Please post back your results.
-Brian
N.,
Thank you for exploring the demo example. To be clear, the particular demo application in the package you downloaded is not designed to work from the QlikView Server. It is intended to be run through the QlikView Desktop. With regard to statconn, please refer to the statconn website for information on licensing of the API for commercial use. The URL is http://www.statconn.com/products.html.
Thank you.
-Brian
The macro is running fine but whatever the value i'm selecting from the patient id list box and running the 'macro run'
button, values are not reflecting on the text objects. What to do now?
My macro is :
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
LOGFILE_NAME = CurrentPath()&"/logs/R.log"
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Sub runRScore
' Export the selected patient data to be scored.
Set myTable=ActiveDocument.GetSheetObject("DataSentToR")
myTable.Export CurrentPath&"/data/ExportFile.csv",",",0
' Create a COM object representing R
Set R = CreateObject("StatConnectorSrv.StatConnector")
R.Init "R"
' Rattle is Copyright (c) 2006-2012 Togaware Pty Ltd.
R.EvaluateNoReturn "library(rattle)"
R.EvaluateNoReturn "building <- TRUE"
R.EvaluateNoReturn "scoring <- ! building"
' The colorspace package is used to generate the colours used in plots, if available.
R.EvaluateNoReturn "library(colorspace)"
' A pre-defined value is used to reset the random seed so that results are repeatable.
R.EvaluateNoReturn "crv$seed <- 42"
' Load the data.
R.EvaluateNoReturn "crs$dataset <- read.csv('"&CurrentPath&"/data/readmission.csv', na.strings=c('.', 'NA', '', '?'), strip.white=TRUE, encoding='UTF-8')"
' The following variable selections have been noted.
R.EvaluateNoReturn "crs$input <- c('enum', 't.start', 't.stop', 'time','event', 'chemo', 'sex', 'dukes','charlson')"
R.EvaluateNoReturn "crs$numeric <- c('enum', 't.start', 't.stop', 'time','event')"
R.EvaluateNoReturn "crs$categoric <- c('chemo', 'sex', 'dukes', 'charlson')"
R.EvaluateNoReturn "crs$target <- 'death'"
R.EvaluateNoReturn "crs$risk <- NULL"
R.EvaluateNoReturn "crs$ident <- c('record', 'id')"
R.EvaluateNoReturn "crs$ignore <- NULL"
R.EvaluateNoReturn "crs$weights <- NULL"
' Decision Tree
' The 'rpart' package provides the 'rpart' function.
R.EvaluateNoReturn "require(rpart, quietly=TRUE)"
' Reset the random number seed to obtain the same results each time.
R.EvaluateNoReturn "set.seed(crv$seed)"
' Build the Decision Tree model.
R.EvaluateNoReturn "crs$rpart <- rpart(death ~ ., data=crs$dataset[, c(crs$input, crs$target)], method='class', parms=list(split='information'), control=rpart.control(usesurrogate=0, maxsurrogate=0))"
' Score a dataset.
' Read a dataset from file for testing the model.
R.EvaluateNoReturn "crs$testset <- read.csv('"&CurrentPath&"/data/ExportFile.csv', na.strings=c('.', 'NA', '', '?'), header=TRUE, sep=',', encoding='UTF-8')"
' Obtain probability scores for the Decision Tree model on ExportFile.csv.
R.EvaluateNoReturn "crs$pr <- predict(crs$rpart, crs$testset[,c(crs$input)], type='class')"
' Extract the relevant variables from the dataset.
R.EvaluateNoReturn "sdata <- subset(crs$testset[,], select=c('record', 'id'))"
' Output the combined data.
R.EvaluateNoReturn "write.csv(cbind(sdata, crs$pr), file='"&CurrentPath&"/results/ExportFile_score_idents.csv', row.names=FALSE)"
' Create plot file for the Decision Tree.
R.EvaluateNoReturn "setwd('"&CurrentPath&"/New Plots')"
R.EvaluateNoReturn "png('R_Score_Plot_1_"&Runtime&".png')"
R.EvaluateNoReturn "plot(crs$rpart)"
R.EvaluateNoReturn "text(crs$rpart, use.n=TRUE)"
R.EvaluateNoReturn "title(main='Decision Tree', sub=format(Sys.time(), '%a %b %d %X %Y'), col.main='blue', col.sub='black', line='2', font.main=4, font.sub=3)"
R.EvaluateNoReturn "dev.off()"
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' Create Bar plot
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' The 'gplots' package provides the 'barplot2' function.
R.EvaluateNoReturn "require(gplots, quietly=TRUE)"
R.EvaluateNoReturn "set.seed(crv$seed)"
' Load the data.
R.EvaluateNoReturn "crs$dataset <- read.csv('"&CurrentPath&"/data/readmission.csv', na.strings=c('.', 'NA', '', '?'), strip.white=TRUE, encoding='UTF-8')"
' Generate the summary data for plotting.
R.EvaluateNoReturn "ds <- rbind(summary(na.omit(crs$dataset[crs$sample,]$sex)), summary(na.omit(crs$dataset[crs$sample,][crs$dataset[crs$sample,]$death=='0',]$sex)), summary(na.omit(crs$dataset[crs$sample,][crs$dataset[crs$sample,]$death=='1',]$sex)))"
' Sort the entries.
R.EvaluateNoReturn "ord <- order(ds[1,], decreasing=TRUE)"
R.EvaluateNoReturn "setwd('"&CurrentPath&"/New Plots')"
R.EvaluateNoReturn "png('R_Score_Plot_4_"&Runtime&".png')"
' Plot the data.
R.EvaluateNoReturn "bp <- barplot2(ds[,ord], beside=TRUE, ylab='Frequency', xlab='Sex', ylim=c(0, 455), col=rainbow_hcl(3))"
'Add the actual frequencies.
R.EvaluateNoReturn "text(bp, ds[,ord]+15, ds[,ord])"
'Add a legend to the plot.
R.EvaluateNoReturn "legend('topright', bty='n', c('All','0','1'), fill=rainbow_hcl(3))"
'Add a title to the plot.
R.EvaluateNoReturn "title(main='Distribution of sex (sample)by death', sub=paste(format(Sys.time(), '%a %b %d %X %Y')), col.main='blue', col.sub='black', line='2', font.main=4, font.sub=3)"
R.EvaluateNoReturn "dev.off()"
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' Close R connection
R.close
Set v = ActiveDocument.GetVariable("vRuntime")
v.SetContent Runtime(), True
' Load the scores into the QlikView application
ActiveDocument.DoReload 2, false,false 'Fail on error, Partial Reload
WriteLog("Retrieved Scores from R at "&Runtime&".")
End Sub
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Function CurrentPath()
Set v = ActiveDocument.GetVariable("vAppPath")
CurrentPath = Replace(v.GetContent.String,"\","/")
End Function
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Function Runtime()
dtmThisDay = Right(Cstr(Day(Date) + 100),2)
dtmThisMonth = Right(Cstr(Month(Date) + 100),2)
dtmThisYear = Year(Date)
dtmThisHour = Right(Cstr(Hour(Time) + 100),2)
dtmThisMinute = Right(Cstr(Minute(Time) + 100),2)
dtmThisSecond = Right(Cstr(Second(Time) + 100),2)
Runtime = dtmThisYear & dtmThisMonth & dtmThisDay & dtmThisHour & dtmThisMinute & dtmThisSecond
End Function
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Function openOutputFile(filename)
'Create and open an output file
Const ForWriting = 2
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set openOutputFile = objFSO.OpenTextFile(filename, ForWriting, True)
End Function
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Function WriteLog(message)
set logFile = openOutputFile(LOGFILE_NAME)
logFile.WriteLine(message)
logFile.Close
End Function
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Here I want to select some id's from the list box and trying to reflect the values as a tree model and as a bar plot on two different text object. The .png files are generating but the value is not reflecting there. I'm giving the Text objects code:
1. Here I'm showing the tree model
=vAppPath&'\New Plots\R_Score_Plot_1_'&$(vRuntime)&'.png'
2. Here I'm showing the Bar plot
=vAppPath&'\New Plots\R_Score_Plot_4_'&$(vRuntime)&'.png'
Thats all. Any suggestion will be helpful.
Apart from this , Is there any other sample code or step by step one page doc. avl. to implement this.. .
This attachment doc 2975 is not helpful unable to open the qvw file in the PE ???
@sathia--- its exactly true. I feel the same. Agree with you.
On Mon, Jul 15, 2013 at 8:05 PM, Samantha Gkonos
Its been long I'm facing the problem. But I have no idea how to solve the porblem. Anybody please help. Thanks in advance.
Hi Brian,
Having the same issue where the module opens up and yes the export encoding has been ANSI by default for me. I am guessing its something to do with the matching versions of rcom, rscproxy, statconnCOM as RandFriends is not available now. Pls could you put the older version file of RandFriends here or maybe mention the matching version nos.
Thanks,
Amit
Hi Brian,
I am unable to see the RAandFriends download at the link provided in the documentation: statconn
Do you know if this has been removed?
Thanks
Dan
Hi Brian,
As with Amit, I am also receiving the Macro Popping up after following the instructions you posted on 25th June above. Any ideas of how we can resolve or is there an earlier version we can install?
------------------------------------------------------------------------------------------------------------------------------
In response to my query above I have:
- download R from: The R Project for Statistical Computing
- statconnDcom non commercial from : statconn
Other Notes when installing the above:
- just before step 10 in the documentation, you will also need to type 'yes' to the prompt in R.You should then see the error as described in step 10.
- At the end rattle also prompted me to install serveral other files three time, i installed all but do not know if this was required or not.
Thanks
Dan
Hi Daniel,
I am facing the same problem as you.Have you found a solution yet?
Hi Brian,
Please have a look to our problems if you have time .
Many thanks.