Statistical and predictive modeling

[#1]

I have a few questions about how people are doing statistical analysis/predictive modeling/data mining with QlikView. According to the Gartner 2010 BI Quadrant, QlikView "lacks the statistical and predictive modeling capabilities of some of its most similar competitors, ..."

My questions:

1) How often are you asked to do predictive modeling with QlikView? (whether you're an in-house developer or work for a consulting firm)

2) What is the most advanced statistical work you've done inside of QlikView?

3) Does anyone use third party statistical packages to pre-calculate before loading into QlikView? (R anyone?)

Thanks,

Gary

[johnw]

Well, this won't be very helpful, but it's AN answer.

1) Never. In house developer.
2) Basic statistical distribution stuff, box plots, 95th percentile, standard deviation, that sort of thing.
3) Not me.

[larsc]

I did some laboration with the rangecorrel function a while ago, but never got the chance to finalize anything.

It could be useful to display the correlation between two (or more) expressions in some scenarios. Let's say number of customers and number of orders.

If the correlation factor between these two expressions is low, the increased amount of orders is probably not a result of the increasing amount of customers, it is probably related to something else.

It may be possible to take this further by analyzing the correlation between more than two expressions, to find the reason behind a trend change in a selected expression.

It is always useful to show a customer the cause behind a trend change in different ways, not only that a trend change exists.

You may play with the rangecorrel function using a line chart with these three expressions:

1) Count(CustomerID)
2) Count(SalesOrderID)
3) rangecorrel(above(total Count(CustomerID), 0, 100),above(total Count (SalesOrderID), 0, 100))

It would be very nice if QlikTech (or someone else) could publish some nice demo's of this, and other statistical functions.

[#4]

John and Lars,

This is exactly what I was looking for, including lack of usage. I am guessing that a majority of QlikView use cases don't include any predictive modeling. 95%? 99%?

I would also like it if QlikTech could promote use cases of predictive modeling within QlikView. Some of our clients are starting to ask about its features and I am investigating the best ways to incorporate it (or not).

Thanks,
Gary

[johnw]

Gary Strader wrote:I am guessing that a majority of QlikView use cases don't include any predictive modeling. 95%? 99%?

99% or higher wouldn't surprise me. Predictive modeling seems likely to be one of the harder things you could do in QlikView. But I see almost no posting on the subject on the forum by people asking for help. So it seems reasonable to conclude that very few people are doing it. Lack of posts on the subject aren't proof, of course, but it's suggestive.

On the other hand, if it IS 99% or higher, it is also strange that you have more than one client asking about it. Guess you're just lucky. 🙂

[#6]

I think you're right about the lack of posts. There are only a handful returned on a forum search.

What's interesting is that while many clients are asking about it, none have actually asked for us to implement it. It is more when they are trying to evaluate whether to purchase QlikView versus the competitor products, they want to know if it has this or that feature. And predictive modeling keeps coming up.

[#7]

I've had a few people ask ABOUT it, (in terms of assessing our capabilities in RFP; we are a QlikView OEM partner) but I have yet to have anyone actually request to implement anything using the tool. The closest I've come is estimating monthly totals based on current daily averages… which I would hardly call predictive analysis. I've also done some 'what-if' type analysis, but again, nothing overly predictive.

Most of the customers that I've implemented it for are blown away by the level of access and insight they gain from QlikView. I think they're overwhelmed by knowing where they've been, or where they've come from to even worry about our ability to predict where they're going, beyond following a trend line.

'Predictive Analysis' is one of those things that people have learned to ask for, but generally have no idea what it is much less how to use it. (Voila! A bicycle for your fish!)

[larsc]

I have actually yet not had any request at all, from any customer that made me have to use any of the statistical functions. But that doesn't mean they aren't powerful, and can't contribute to the QlikView experience. I'm sure they can!

For example, such a thing as finding the most probable cause behind a trend change using the rangecorrel function. There must be many areas where this can be useful, implemented in a "user friendly" way. The problem is, using these functions might need some explanation depending of the end users. Not everyone knows what a correlation coefficient is

Maybe all we can do is to start a group in this forum, where we who are interested in the statistical functions can start to develop and share some stuff together!

[#9]

Hi,

We deal with call volumes and most of the modeling is done through another application, which we tap into and pull finished results via the View made available. However, we have on occasions tried to make some predictive figures based upon historical events and took the mean figures to apply into the future. It's not exactly Erlang, but in a short period it can make fairly accurate projections. Our big buzz just now is using Distribution curves and I have felt Qlikview as a tool fails to deliver thus far. This maybe our building that is the failure, but either way we have not found Qlikview the reliable tool in this area to date.

I would be happy to see if anyone has better success as we use Qlikview heavily for repeatable reports and it's likely useful for much more.

Thanks

Peter

[danielrozental]

1) Very rarely. Consultant

2) Calculating a linear regression, a colleague did a Beta function calculation.

3) I've used R to calculate a multiple linear regression, but it was just for a prototype. Batch offline integration should be quite possible using command line instructions, not so sure about interactive analysis though.

Will definitely love QT to add more features regarding this but really doubt they will.