User selection and parameter for SSE Plugin

[virilo_tejedor]

Hi,

I have translated the Kmeans example to Python, using HelloWorld code as reference.  I have this expression working:

IrisPython.PredictIris([petal length], [petal width], [sepal length], [sepal width])

Now, I'd like to pass the parameter n_cluster to the python SSE plugin.

If I do the next: IrisPython.PredictIris(3, [petal length], [petal width], [sepal length], [sepal width])
The value "3" is going to be replicated on each row.  It would be very inefficient using some parameters like that for a large dataset.

Also, I'd like to let the user to chose the n_cluster value. 

Perhaps, it could be possible using a new data field:

LOAD * Inline
[NUM_CLUSTER
1
2
3
4
5
6
7
...
];

Could the SSE plugin know the user selection?

How could I pass parameters in addition to data to a python script?

Thanks in advance,

Virilo

[Anonymous]

Hi virilo.tejedor,

We are aware of the limitation of not being able to send constants as parameters, without it being replicated on each row as you said, and it's in our backlog. As of today the data passed as parameters to the plugin must have the same cardinality. When you pass a field to the plugin, Qlik will send the selected values in that field, to the plugin. If no selection was made, all values will be sent.

You can create a field for the number of clusters and let the user make selections, but then you need to handle the fact that the user might choose more than one value. I would recommend you to use a variable instead. If you're using a plugin defined function, like the PredictIris function, you have to pass the variable as a parameter. But if you are using a script function you can use string concatenation to include the variable directly in the script without having to pass it as a separate parameter.

[Anonymous]

Josefine described it well.

Just to give you an example of using a variable as constant when calling scripteval:

If myvar is my variable, then you can pass it in the script string as $(myvar), see below example:

Script.ScriptEval('list(numpy.asarray(args[0]) + $(myvar))', Numeric)

There also exists another way, still a bit hard for the one writing the expressions in Qlik but it is possible at least until the needed functionality is added in Qlik. You can expose your own Iris-methods on the python side so that they are accessible from scripteval calls. In that case you can call your iris-methods through a scripteval call like below example shows where I call my method helloworld:

Script.ScriptEvalStr('myfuncs.helloworld(str($(myvar)), args[0])', String)

However you need to change the python plugin to expose your methods. In my example I have modified the script example (ScriptEval_script.py) by first adding a new class that contains my own method like below:

class MyFunctions:

    @staticmethod

    def helloworld(conststring, mystrings):

        return iter([str.join(conststring) for str in mystrings])

Then I add the following line in evaluate method:

funcs = MyFunctions()

and finally I pass that class in the eval call next to the other exposed classes:

        result = eval(script, {'args': params, 'numpy': numpy, 'myfuncs': funcs})

I know, it is not nice, but could be worth trying.

[virilo_tejedor]

Thanks Josefine, Tobias for your responses.

I tried this simple example:

=Script.ScriptEval('list(numpy.asarray(args[0]) + 5)', [petal width])

But I'm receiving a nan as arg[0].  I added some extra traces:

2018-01-19 13:29:16,296 - INFO - Logging enabled
2018-01-19 13:29:16,328 - INFO - *** Running server in insecure mode on port: 50600 ***
2018-01-19 13:29:45,870 - INFO - EvaluateScript: list(numpy.asarray(args[0]) + 5) (ArgType.Numeric ReturnType.Numeric) FunctionType.Tensor
header.params !
Evaluate script row wise
call to evaluate
    ----------------
script:  list(numpy.asarray(args[0]) + 5)
params:  [nan]
ret_type:  ReturnType.Numeric
2018-01-19 13:29:45,920 - ERROR - Exception iterating responses: 'numpy.float64' object is not iterable
Traceback (most recent call last):
   File "C:\Users\virilo.tejedor\AppData\Local\Continuum\Anaconda3\lib\site-packages\grpc\_server.py", line 393, in _take_response_from_response_iterator
     return next(response_iterator), True
   File "C:\POC\src\SSE_Plugins-0.1\FullScriptSupport\ScriptEval_script.py", line 59, in EvaluateScript
     yield self.evaluate(header.script, ret_type, params=params)
   File "C:\POC\src\SSE_Plugins-0.1\FullScriptSupport\ScriptEval_script.py", line 177, in evaluate
     result = eval(script, {'args': params, 'numpy': numpy})
   File "<string>", line 1, in <module>
TypeError: 'numpy.float64' object is not iterable

Why am I receiving a NaN?

Also, it seems like doing it in this way is going to perform an evaluate execution per row (Evaluate script row wise)

It won't allow me some use cases, like retrain the model using the selected data or perform a moving average

As workaround I'm exposing another funtion SetUserParam to send the user variables to python; and qsVariable for the selectors in the UI.

This workaround have some issues:

- race conditions with other requests
- the selectors aren't refreshing the graphs due to Qlik doesn't know that Y hat is modified in SetUserParam call

Since it is a proof of concept for using Advanced Analytics in Qlik, I could wait for future versions without this limitation.

Thanks again!

[Anonymous]

Hi Virilio!

It's hard for me to say exactly why you receive a NaN without knowing how your data model looks like and what plugin you're using. If you could provide the .qvf file I could take a look. Did you use the FullScriptSupport example when you tried to run `=Script.ScriptEval('list(numpy.asarray(args[0]) + 5)', [petal width])`?

We released a new SSE version yesterday (v1.1.0) where we also updated the python examples, one of the updates being to evaluate the script after all data is collected(and not per row as you noticed), in the script example. There is also a new python script example using pandas and exec, which is better suitable for more complex scripts. Read more about the pandas example here. Note that the new features in the SSE protocol v1.1.0 are supported first in Sense February 2018.

[Anonymous]

Just a quick update:

I tried using the latest version of FullScriptSupport and the Ctrl+00 script, and the following expression worked fine:

=Script.ScriptEval('list(numpy.asarray(args[0]) + 5)', AsciiNum)

If you are using the provided examples, I would recommend you to update to the latest version and try again, to see if you still have the same issue.

Let me know how it goes!