topic Re: API for in-memory access? in QlikView

API for in-memory access?

Fri, 10 Apr 2015 16:11:24 GMT

Hi All -

Apologize for asking what may be a known to you all, but does Qlik provide API(s) for directly accessing the in-memory data store?

Thanks in advance!

Brian

Re: API for in-memory access?

Gysbert_Wassenaar — Sat, 11 Apr 2015 09:18:03 GMT

Not for Qlikview afaik. There is an API for the Qlik Sense Engine that allows you to calculate with and retrieve data from the in-memory data store of Qlik Sense apps.

Re: API for in-memory access?

petter — Sat, 11 Apr 2015 14:40:56 GMT

You can certainly gain access to the in-memory data of a running QlikView application both for an application you are running in single-user or on a server shared by many.

First there is the application's own internal Module/Macro that can be written in either VBScript or JScript. It use the QlikView Object Model to gain access to any part of the QlikView enviroment and data.

The QlikView object model expose a rich - but not very fast - API. This can be leveraged not only from the inside as mentioned first but also from the outside by any scripting or programming language that supports Windows Automation. So everything from a simple VBscript, JScript running from a .vbs/.js text-file external to the application and Microsoft Office (through VBA) to .NET languages or Python can connect to a .QVW (single-user) or to a server via the QVP-protocol. Any of these programming languages can control and automate QlikView in the way a user would do interactively and hide or show QlikView while it does so. The control is more extensive than what a user can gain via the user-interface.

Doing massive manipulation of data with Windows Automation without resorting to reloads is rather slow but apart from that you can really gain full control of the QlikView-application and integrate it with other applications that way.

Via the JavaScript API you can also gain access and integrate with QlikView server-applications in a number of ways through HTTP. It is not only limited to writing object extensions and document extensions.

And there is something called Dynamic Update and it can be used in several ways to change/update in-memory data. It is not well documented and it has some serious limitations - but as long as you stick to some sensible rules and thread very carefully it can be useful. The documentation is very slim. You can leverage it even in a regular QlikView applications by using Buttons/Text Boxes/Actions and doing INSERT, UPDATE and DELETE against the in-memory data-model. This can be employed from any Macro Module too (VBScript/JScript) and the external Windows Automation. Qlik themselves does not promote this and it might disappear in future version of QlikView. It is not supported in Qlik Sense.

There is an OCX-version of the Windows Automation too - which is mostly or maybe only used by OEM-partners.

Hope this is not too confusing - but if you have a more specific need I could certainly elaborate a little bit more on a specific approach...

HTH

- Petter

Re: API for in-memory access?

petter — Sat, 11 Apr 2015 14:48:32 GMT

There is a lot of information to be found here on the community - for instance here:

QlikView Version 11 SDK

Along with the proper installation of "Documentation and Tutorial" which is separately and freely downloadable for anyone you will get a QlikView Application that is a comprehensive documentation of the Windows Automation and OCX-API. It is called APIGuide.QVW (AFAIK last updated this January).

Re: API for in-memory access?

Mon, 13 Apr 2015 13:06:30 GMT

Hi Petter –

Thanks for the responses. Unfortunately, we require direct API access because OCX, as you mentioned in your note, will create a performance issue that we cannot absorb in the architecture we are putting together.

Thanks,

Brian

Re: API for in-memory access?

petter — Mon, 13 Apr 2015 15:55:23 GMT

Then you probably should go for an in-memory external DB-solution that you can connect to QlikView via Direct Discovery. There are several in-memory closed-source and open-source solutions to pick amongst.

One of the newest partners to Qlik that connect quite well via Direct Discovery is JethroData. This leverage Hadoop and creates a solution that can be used in close integration with QlikView and Qlik Sense.

Re: API for in-memory access?

petter — Mon, 13 Apr 2015 15:58:41 GMT

Look here: Qlik Sense‌ and JethroData demo-site

Re: API for in-memory access?

rwunderlich — Mon, 13 Apr 2015 22:44:23 GMT

A clarification. For QlikView, you can access the fieldvalues via API but you cannot access data tables row by row. The workaround is to create a chart or tablebox that contains the rows and fields you want. The API can read rows of a sheet object.

-Rob

Re: API for in-memory access?

Tue, 12 May 2015 13:48:39 GMT

Hi Rob,

Does QlikSense differ from QlikView in this regard? Is it possible to access data tables directly in the Qix database via the Qlik Engine API?

Thanks,

alessandra

P.S. Sorry that this thread is about QlikView not Qlik Sense but I am exploring access to data in both.

See my questionWhat are exactly Structs objects for?

Re: API for in-memory access?

rwunderlich — Tue, 12 May 2015 18:37:24 GMT

You can create hypercubes -- dimensions & measures -- on the fly in Sense and then access that data. I don't know if direct table access is provided, but you can simulate the same with a hypercube.

-Rob

Re: API for in-memory access?

Thu, 18 Jun 2015 15:02:34 GMT

Hi Peter/Rob,

That's an amazing piece of information you have provided there.

Now, regarding the points you have mentioned above, there's a need for us to push data obtained on the fly into an extension from a web service into a table in the data model. So basically a dynamic data update from JavaScript. Also, would the data updated by dynamic update persist for eternity or is it only for that user during that session?

Any pointers in this would be very helpful. Or any other workaround that you can suggest also would work.

Thanks,

Re: API for in-memory access?

rwunderlich — Thu, 18 Jun 2015 19:25:13 GMT

KR, QlikView or Qlik Sense?

-Rob

Re: API for in-memory access?

Fri, 19 Jun 2015 09:31:43 GMT

Hi Rob, this is for QlikView.

Thanks,

Re: API for in-memory access?

rwunderlich — Mon, 22 Jun 2015 04:56:53 GMT

QlikView has the Dynamic Data Update facility. Your extension could fetch new data and insert it using the DynamicUpdateCommand API call. I have not thought through how/if an extension can call the API. If there is not an extension mechanism to invoke the API directly, perhaps the extension can simulate a button click that runs the macro. Sorry for the incomplete solution, but hopefully will get you pointed in a direction. Let us know if you figure something out. rbecher‌ is my goto guy on extensions, perhaps he can suggest an approach.

-Rob

Re: API for in-memory access?

Mon, 22 Jun 2015 09:18:13 GMT

Hi Rob!

Thanks for the points you mentioned and guiding in the right direction. I am not actually finding the way to invoke the API from the extension as this is in JavaScript and in AJAX/WebView mode, wherein, the button triggering macro or simulating a macro call is not working.

I understand from above and from various other community posts and DOC-XXXs that there would be QlikView JavaScript API to manipulate the data model,(like the one that you also mentioned 'DynamicUpdateCommand API') but am not able to find the right invocations/parameters to be used. Have gone through the JavaScript API doc by QlikView and help files, etc., but no luck.

If there's something in this space that one can help would be really great. Also, this‌ blog post says there is no such API provided for JavaScript based extensions as of now. Not sure if it is still not available or only the documentation is not updated.

However, post success of this small poc, I shall be sharing the way how this is done in a blog post that am working on and hoping it would be a useful resource.

Hi rbecher‌ rwunderlich‌ stephen-x.redmond‌, swr‌, hic‌, dgudkov‌, any inputs further shall be really helpful.

Thanks,

Re: API for in-memory access?

rbecher — Mon, 22 Jun 2015 09:59:39 GMT

Hi knightriderX,

the only way I know (also mentioned in the blog post from dgudkov you've linked) is the variable change action (plus update macro) you could trigger from an extension. But I wonder what use case you want to accomplish. You should give us more details.

Extensions run in a user session only so data ingestion would be available only there, dependent from user interaction. QlikView isn't probably the right tool for this requirement. I would ask why you would need the data in the QlikView data model itself. Sometimes a data blending is a good way for integration (and easy to do): having data rendered from another system via Ajax call in an extension. So the user has this external data in a UI component besides the QlikView loaded data. You could also use the external data to make a selection in QlikView.

- Ralf

Re: API for in-memory access?

Wed, 24 Jun 2015 11:22:17 GMT

Hi Ralf,

Thanks for the quick reply. Due to confidentiality reason, am not able to delve into the actual requirement details.

I understand about the point you mentioned:

Extensions run in a user session only so data ingestion would be available only there, dependent from user interaction.

However, on the standpoint of the requirement, it is sufficient to have data obtained from the web-service be getting updated into the data model only for the user session and only for the user based on the user privileges(the web-service takes care of this).

On your point :

Sometimes a data blending is a good way for integration (and easy to do): having data rendered from another system via Ajax call in an extension.

Yeah, i completely agree on this as it might be like a near-real-time analysis.

I have a working prototype of this wherein one can select YHOO or APPL or both in a field loaded in QlikView and the extension invokes the web-service for the values selected and fetches Stock values accordingly. Now this is available inside the extension and can be displayed.

The challenge faced now is that, from within the extension the table in the data model should be updated with the new stock values against those codes. I see no API or documentation that could let me do this using JavaScript.

And I have a feeling that I have not clearly understood this point of yours:

Sometimes a data blending is a good way for integration (and easy to do): having data rendered from another system via Ajax call in an extension. So the user has this external data in a UI component besides the QlikView loaded data. You could also use the external data to make a selection in QlikView.

Please do let know if you have further suggestions or can guide me in a better direction.

Thanks,

Re: API for in-memory access?

petter — Wed, 24 Jun 2015 15:27:16 GMT

A "Dynamic Update" can be invoked by calling the click method of a button object. You will have to define a button and associate an action with it that is a "Dynamic Update". The Dynamic Update can use as a parameter a variable which contains the dynamic update commands: delete, insert, update, transaction etc ...

Variables can be set from the JavaScript API and buttons can be clicked programatically also.

{void} Click()

Note! Only available for Buttons.
Trigger a click on the button.

    qvButton.Click();