Server Side Extension (SSE) Limitation

[morenoju]

Hi folks,

I've started to use server side extensions to allow my Qlik Sense apps to access a REST API and manipulate the response in Python before displaying results on Qlik.

However, I've run into a limitation that is blocking my development. I need to be able to create functions that receive one argument (an ID) and return a list (one column with several rows). My Python code is already providing the desired list for each of the ID values that I pass to the function through Qlik, however, I've noticed how Qlik receives the first row of the result column and discards the rest of the rows. I've seen in the documentation that this is an existing limitation.

Do you know of any way to work around this?

From: https://github.com/qlik-oss/server-side-extension/blob/master/docs/limitations.md

Expressions using SSE must persist the cardinality

When you use SSE in a chart expression or in the Qlik load script (the LOAD ... EXTENSION ... statement excluded), you should preserve the cardinality and return a single column. In the case of aggregations, the response column should contain a single value (one row). In the case of tensor functions, the response column should contain the same number of rows as the request.

• If too little data is returned from the plugin, Qlik will add null values to match the number of rows expected.
• If higher-order data is returned, either too many rows or too many columns, Qlik will discard any additional data.

[rwunderlich]

You can return the rows concatenated with a delimiter (e.g. ; ) which you then break into rows using SubField(). 

Load SubField(MYSSE.MyFunc(ID), ';') as newdata

Or use the Load Extension syntax.  Be aware that the Load Extension syntax does not allow literal parameters so you have to load the ID to a temp table first. 

-Rob

[morenoju]

Hi @rwunderlich , thanks for your response.

I actually tried that approach (rows concatenated and then SuField), however, the requirement I'm trying to fulfill in my app needs the request to the REST endpoint be made from the chart itself (in execution time, not in the loading script). I'm supposed to provide a very "dynamic" app, where users select one ID and then new data (a column, a table) "comes into the app" right away as part of the response of the SSE function. That's why I can't find a way to workaround the cardinality limitation.

Do you have any idea to address this in a chart instead of in the load script?

Thanks

[rwunderlich]

You could make the chart itself an extension and then it can decode the results.  But if you do that, you can bypass the SSE entirely and make the REST request directly from your extension. 

Now that I understand you are doing this in a chart, I'm curious about something. You mentioned returning multiple "rows". In the context of this chart do you mean multiple display rows that map to a single Dimensional rows or are you trying to generate new Dimensional values in the chart?  Is this like expanding a Pivot table level?

-Rob

[morenoju]

What I'm trying to achieve is that every time the user selects a different ID from a filter pane, the chart (for now, a table) displays the table returned by a REST endpoint.

Since I expect the number of columns and headers to be fixed, I can be OK with bringing the results column by column (defining each dimension as a call to my SSE function). But I definitely need more than one row in the response even if the input is only one item (the ID).

Regarding bypassing the SSE, I wonder if the difficulty might be to retrieveing and parsing the nested JSON returned by the REST endpoint in the Javascript code of the extension. In SSE I'm using Python but my understanding is that visualization extensions have to be written in Javascript.

[morenoju]

Adding a bit of more context: My app currently works because in the data load script I'm calling the REST endpoint in a loop for all possible ID values. That way I retrieve the results, store them in a table in memory, and I can display them depending on the user selection of ID.

However, this is not going to scale well when I have thousands of IDs. I would needto instead of calling the REST endpoint from the script, calling them from the chart only when a new ID has been selected. That way I will be able to avoid overwhelming the memory.

Within a visualization extension, would it be possible to send a REST request every time the ID selection changes and generate a "results" table with columns and rows (as many as returned by the REST service) where I would store the results read from the REST response?

[rwunderlich · Accepted solution]

"Within a visualization extension, would it be possible to send a REST request every time the ID selection changes and generate a "results" table with columns and rows (as many as returned by the REST service) where I would store the results read from the REST response?"

Yes. Your extension's Paint method would get called every time the ID selection changed. You would then fetch the new data and update your display.  Some things to keep in mind. 

1. HTTP requests and parsing JSON in javascript is super easy.

2. Your visualization extension is responsible for creating the entire visualization.  You are not mixing a column into a native viz like you are with an SSE function call. If your requirement is just an HTML table with values, this is simple.

3. Your extension does not necessarily need to call the source REST endpoint directly.  For example, if you already had some python code you wanted to re-use, you could wrap that python code in a webservice and call that.  You would need to provide the webservice/python environment so it may not be worth the trouble. 

-Rob