My friend,Øystein Kolsrud - Software Architect at Qlik, is back with part 2 of the Qlik Engine API fundamentals.IntroductionDuring Part 1 of this series, the focus was on the basic JSON-RPC aspects of the engine API.This post will focus on a part of the API that is engine specific and fundamental to how the engine API works: Handles.The "Global" handleLet's go back to the call to "QTProduct" that was used as example in Part 1. The request object for that method looks like this:{ "jsonrpc": "2.0", "id": 0, "method": "QTProduct", "handle": -1, "params": []}The four properties "jsonrpc", "id", "method" and "params" are all part of the standard JSON-RPC protocol, but the fifth one, "handle" is not. The engine API uses this property to define the context in which the method should be executed. '-1' is a special handle that identifies what is called the "Global" context which is a context that is associated with the Qlik Sense system it self. The methods available in this context are all relates to system wide behavior such as properties of the server where the engine runs. It is also the entry point for accessing Qlik Sense functionality, and arguably the most important endpoint in the global context is the method "OpenDoc". This endpoint tells the engine to make a specified app ready for interaction, which means loading the app from disk into memory on the server (if it is has not already done so). The specification for this method can be found here:https://help.qlik.com/en-US/sense-developer/June2020/APIs/EngineAPI/services-Global-OpenDoc.htmlAnd a typical RPC request object for this method can look like this:{ "jsonrpc": "2.0", "id": 1, "method": "OpenDoc", "handle": -1, "params": ["eca1e963-d9e1-405c-9262-1179f19f0de3"]}The handle is set to '-1' to ensure that the method is executed in the global context, and the value passed as parameter is the ID of the App that is to be opened. (The name "OpenDoc" stems from QlikView where the entity corresponding to a Qlik Sense App is called a Document.)Working with handlesUpon successfully opening the requested app, the server will respond with a response object that looks something like this:{ "jsonrpc": "2.0", "id": 1, "result": { "qReturn": { "qType": "Doc", "qHandle": 1, "qGenericId": "eca1e963-d9e1-405c-9262-1179f19f0de3" } },"change": [1]}The value of the property "qReturn" is an object of the type "ObjectInterface" as defined in the engine reference documentation for the method. The most important member of that object is the one named "qHandle" which informs the client that the engine has assigned a handle to the newly opened app and that this handle should be used when interacting with the engine in the context of that app. If we for instance want to evaluate the sum of sales for the app, then we can use the following method:https://help.qlik.com/en-US/sense-developer/June2020/APIs/EngineAPI/services-Doc-Evaluate.htmlThe request object for this method would look like this:{ "jsonrpc": "2.0", "id": 2, "method": "Evaluate", "handle": 1, "params": ["Sum(Sales)"]}Notice that the "handle" property is now set to '1' instead of '-1' which was the case for previous calls. It does not make sense to run this method in the server context, as all evaluations must be done in the context of an app. And correspondingly, calling the "OpenDoc" method with the handle '1' would not work, as it does not make sense to open an app in the context of an app. If you try to do so, then the engine will simply return an error.But given that the app was successfully opened and that there is indeed a field with the name "Sales" in that app, then the engine would respond to the "Evaluate" method with an object like this where the "qReturn" value is the result of calculating the sum of sales:{ "jsonrpc": "2.0", "id": 2, "result": { "qReturn": "240491" }}Our first selectionThe above call to "Evaluate" computed the sum of sales for the app. However, it is typically of more interest to perform computations given a specific subset of the data of the app, for instance per sales person. There are many ways to accomplish this in Qlik Sense, but I will in this example first perform a selection in a specific field, and then use the "Evaluate" method.To perform a selection in a field, we will first need to get a handle to use for expressing the context of the field where we want to perform the selection. This is exactly the purpose of the method "GetField":https://help.qlik.com/en-US/sense-developer/June2020/APIs/EngineAPI/services-Doc-GetField.htmlA call to this method would look like this:{ "jsonrpc": "2.0", "id": 3, "method": "GetField", "handle": 1, "params": ["SalesRep"]}And the engine will reply to this message by sending a new object interface for the field named "SalesRep":{ "jsonrpc": "2.0", "id": 3, "result": { "qReturn": { "qType": "Field", "qHandle": 2 } }}We can now use the handle '2' for interacting with this field, and I will use the method "Select" to perform a selection of a particular employee:https://help.qlik.com/en-US/sense-developer/June2020/APIs/EngineAPI/services-Field-Select.htmlThe call will look like this, and notice that the handle is now set to '2' as that was the handle we got in response to the "GetField" method:{ "jsonrpc": "2.0", "id": 4, "method": "Select", "handle": 2, "params": ["Amalia Craig"]}If the selection is executed successfully, then the engine will reply with the return value 'true' like this:{ "jsonrpc": "2.0", "id": 4, "result": { "qReturn": true },"change": [1, 2]}Selection stateThe "Select" method has an important side effect, namely to change the state of the app to reflect that a certain value has been selected. This was, of course, the whole purpose of performing the call in the first place. The property in the response named "change" contains a list of handles that might have been affected by his action, and in our case, the affected handles are '1' which is the handle for the app, and '2' which is the handle for the field where we performed the selection. Now that we have performed the selection, we can call the "Evaluate" method in this new state:{ "jsonrpc": "2.0", "id": 5, "method": "Evaluate", "handle": 1, "params": ["Sum(Sales)"]}Which will return us the sum of sales computed for the sales rep "Amanda Craig":{ "jsonrpc": "2.0", "id": 5, "result": { "qReturn": "27552" }}SummaryHandles are used in the engine API to provide method context. Methods such as "OpenDoc" and "GetField" return a handle that the client can use to access functionality associated with the requested object. These handles are used both by the client to request operations on the server, but also by the server to inform the client that certain events have occurred that affects the state of the objects the handles represent.
...View More
Today our special guest blogger isØystein Kolsrud -a Software Architect at Qlik, part of the Customer First team which serves as a bridge between support and R&D. He is one of the main developers behind the .NET SDK, and has extensive experience with the Qlik Sense APIs.Øystein has prepared a series of technical posts that outline the Qlik Engine API fundamentals. That we will release over the next coming weeks in the Qlik Design Blog.Part 1 covers JSON-RPC fundamentals. Please post your comments and questions below and I'll make sure thatØystein is aware so he can respond appropriately.--------------------------------------------- Over the years I have answered a number of questions on Qlik Community relating to the Qlik Sense APIs. My main focus has typically been on the .NET SDK, but many times my answers to questions have not really concern the .NET implementation all, but are in fact relevant for all interactions with the engine, no matter what technology is used for the communication. With this and future blog posts, I will try to explain some of the fundamentals of how the engine API works. My hope is that these posts will be of interest to anyone who would like to become more familiar with the inner workings of the engine API. And though most libraries used for handling engine communication do their best to hide many of the low level concepts I will describe, it is not uncommon that a certain behavior is difficult to explain without going back to the basics. At least, I hope I will in the future be able to reference to these posts as the need arises. So let's start, and the first thing I'll do is to go back to the absolute basics: JSON-RPCJSON-RPCRPC is short for "Remote Procedure Call", and JSON (much like XML) is a way to structure data as strings. I will not be covering more information about JSON, as there are plenty of resources on the web for doing that, suffice to say that a well structured JSON string can take any of four types: String, Number, Array or Object. A JSON object consists of a set of properties that have a name and an associated JSON string, while an array is simply a list of JSON strings. The very first thing that happens when you connect to the engine is that you will receive a message from the engine with the following text over the websocket:{ "jsonrpc": "2.0", "method": "OnAuthenticationInformation", "params": { "userId": "demouser_1", "userDirectory": "DEMO", "logoutUri": "https://localhost/demo/qps/user", "serverNodeId": "7183c034-716e-4b68-844e-c91270a87148", "mustAuthenticate": false }}This message is a JSON object describing a method call, or a "Remote Procedure Call" if you like. It is a request sent from the server to the connecting client telling it to perform a particular operation. The three top level properties in the message are all part of the standard specification for the JSON-RPC protocol and are not Qlik-specific constructs. In fact the whole message conforms to the JSON-RPC Request object specification of the JSON-RPC specification. More details about that standard can be found here:https://www.jsonrpc.org/specificationThe first property named "jsonrpc" simply specifies which version of the JSON-RPC protocol that is being used and will always by "2.0" when interacting with Qlik Sense. The two other properties, "method" and "parameters", contain information about what operation the sender is requesting the receiver to perform. Now it is totally up to the receive to choose what do do with this message, but the most important information in this specific call is the "mustAuthenciate" property of the parameters, which, as the name implies, states whether or not the client must take any particular actions in order to become authentication. The fact that the value is "false" here is a good thing as it means that we are all good to go and ready to start sending requests to the server.Note that the values of the parameters of the "OnAuthenticationInformation" call will differ depending on what authentication mechanism is used to connect to the server. The particular format show above is what you get when you connect successfully through a virtual proxy using static header authentication. I will not delve deeper into the authentication flow in these blog posts, as that is a quite big area and would probably benefit from a set of blog posts on its own. I chose static header authentication for this post simply because it is usually one of the easiest ones to get up and running.Sending our first messageNow that we have an open connection to the engine, we are ready to send our first message. When connection through the .NET SDK, the first message to be sent will always be a particular RPC that simply ensures that the connection is open. This is necessary since, for some authentication methods, the "OnAuthenticationInformation" message will be sent by the server only after a valid message has been received from the client. Doing this first call therefore has the effect that we capture any authentication failures as early as possible after opening the connection. The call looks like this:{ "jsonrpc": "2.0", "id": 0, "method": "QTProduct", "handle": -1, "params": []}We have already discussed the "method" and "params" properties in the previous section, and their values in this case conform to the specification of the engine API method called "QTProduct" as defined in the engine API reference documentation:https://help.qlik.com/en-US/sense-developer/June2020/APIs/EngineAPI/services-Global-QTProduct.htmlBut the call also introduces two new concepts that are very important to understand how the engine API works. The first of these concepts is the "request id".RPC request IDThe "id" property of an RPC request object is used as an identifier to connect an RPC request with a response. Upon receiving an RPC request, the engine will start processing that request, and when it is done, it will send back a response to the client with the result. This response will contain the same value for the "id" property as the one used in the request. For the "QTProduct" request above, the response will typically look like this:{ "jsonrpc": "2.0", "id": 0, "result": { "qReturn": "Engine 64-bit Edition (x64)" }}The contents of this message is called an "RPC response object" and also conforms to the JSON-RPC specification. Note that the value of the property "id" is the same as the value used in the request in the previous section. This is important as specifying the id makes it possible to have multiple request active in parallel without worrying about the order in which the responses are returned. We could for instance send the two following requests to the engine:{ "jsonrpc": "2.0", "id": 0, "method": "QTProduct", "handle": -1, "params": []}{ "jsonrpc": "2.0", "id": 1, "method": "OSName", "handle": -1, "params": []}The engine would then start processing these in parallel and reply in whatever order the processing completes. So the responses could very well come back in reverse order like this:{ "jsonrpc": "2.0", "id": 1, "result": { "qReturn": "WindowsNT" }}{ "jsonrpc": "2.0", "id": 0, "result": { "qReturn": "Engine 64-bit Edition (x64)" }}It is up to the client to connect the responses to the correct requests based on he "id" property, and handling this is one of the key things that libraries like Enigma and the .NET SDK take care of. A user of such libraries can safely perform concurrent operations and let the libraries take care of the low level matching of requests and responses due to this "id" property.I mentioned earlier that the call to "QTProduct" introduced two important concept out of which the "id" is the first. The second important concept is the "handle", and handles will be the primary topic of the next blog post in this series.
...View More
With the June 2020 release of Qlik Sense came many chart enhancements. My favorite is the addition of mini charts to a table. In a table, a measure can be visualized via a mini chart as either bars, dots, sparklines, or positive/negative.This enhancement not only makes a table more appealing to look at, but it can easily point out changes and trends in the data. To add a mini chart to a table, follow these simple steps:Add a measure to the table. In the example above, Sum(Sales) is the measure.In the Properties window of the table, change Representation to Mini chart.Select the dimension to indicate what data to view. In the table above, Month was the selected dimension.Select the Type of mini chart to add (bars, dots, sparklines or positive/negative).That is it, the mini chart is added to the table. There are other options that can be adjusted to change the look and feel of the mini chart. In the Colors area, there is the option to select the color of the bar/dot/line as well as the option to select the color of the max value, min value, first value and last value. If showing a sparkline mini chart, you also have the option to show dots on the line for each data point of the selected dimension. The y-axis can also be adjusted for the mini chart. There is the option to use local or global so you can decide if the y-axis range is based on the specific row or all rows. There is also the choice to select auto, zero as baseline or zero as center for the y-axis. Read more about mini chart in Qlik Help.Mini charts are eye candy for your table. Dress up the measures in your table with mini charts to provide another layer of information in an easy to digest manner. Learn more about the Qlik Sense June 2020 release by checking the links below:Qlik Sense App - What’s New AppVideo – What’s New – Qlik Sense June 2020Video – June 2020 Feature DemonstrationBlog – Qlik Product Innovation Blog – Qlik Sense June 2020Thanks,Jennell
...View More
Join me and my special guest Jeff Goldberg, Product Manager Integration and APIs, this Thursday at 2PM ET to learn about our new Command Line Interface for Qlik Sense SaaS - qlik-cli
A few months ago I stumbled upon a fantastic app that showcase the not very well known math functions in Qlik. The app does a great work popularizing math & statistics using default Qlik functions. Ever since I put my hands on the app I wanted to blog about it.Explore the app here:https://showcase3.qlik.com/sense/app/c863e07b-87bc-4d0f-a6e1-92f3289b4f56More info and app download here:https://community.qlik.com/t5/Qlik-Sense-Documents-Videos/Math-amp-Statistics-in-Qlik-Sense-app-included/ta-p/1665018I asked the app's author, Mária Šándorová@JaMajka1to introduce the app:I started to develop the app like this the same year I started to work as a full-time Qlik developer. For me it was a good way how to explore mathematical possibilities together with script and expressions syntax of a great tool that was new to me. After that it didn’t have my focus for some time. And then there was a boom of advanced analytics and predictions. Qlik community, different blogs, even my own presentations and Qonnections were full of SSE – possibilities how to utilize mathematical power of R or python for these algorithms. They are great tools with their advantages in more complex algorithms and a good step forward for Qlik that it can be integrated with them. However, I think that because of all these information and materials about advanced analytics for Qlik in R or Python or anywhere else, we started to underestimate Qlik’s own capabilities. When someone needs to calculate a correlation, identify outliers, run k-means clustering or test data normality, we really don’t need to use an external tool – we can use Qlik’s default functions as are or script an algorithm within Qlik. I spent days building this app and I still haven’t covered many areas like hypothesis testing or linear regression. Not to mention algorithms you can script within Qlik. So there really are many advanced analytics and statistics possibilities integrated within Qlik itself, too. I think it’s amazing that we have different options on how and where to calculate something! And I believe that for being able to make a good choice, we need to consider our options and the first step is to know we have it 😊. So, exploring Qlik’s capabilities in statistics was the first reason to create this app. It defines the content of it. The structure and design of the app is driven by something different. I really like data literacy initiatives and I believe we need to help people become data intelligent and understand their data. I know, this app is more about mathematics than about context and visualizations, but I think it’s also an integral part of the data intelligent company – even if not necessary important for all users. And what is a better way how to understand the function if not having a simple use case and a generated dataset in an interactive tool? I love the idea of trying possibilities and seeing the results in the same second. In advanced sections of the app you can select a subset of data in two clicks and thanks to Qlik, everything is recalculated! Select outliers only and see the results of mean and variance … select excluded values and see their results… That’s brilliant for understanding what’s going on! And if you prefer specific formulas and definitions, they are there, as well 😉.I hope you like it as much as I do
...View More
Join me this Thursday to learn about my new webinar series.Sign up here:https://go.qlik.com/Do-More-with-Qlik-Webinar-Series?_ga=2.59309549.1553862558.1591019803-2011899493.1579200524
...View More
.png "Former Employee"){kind=icon}
