topic Re: Can't perform a simple Engine API call to an app using python in Integration, Extension & APIs

Can't perform a simple Engine API call to an app using python

virilo_tejedor — Tue, 08 Aug 2023 10:22:07 GMT

Update 2023.08.08 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Solved, Thanks to Øystein Kolsrud! (accepted solution)

Here you can find the python working code to perform a simple WSS call to Engine API

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I'm trying to perform a very simple Engine API method call using python websockets-client.

My setup:

- Qlik Sense Server Enterprise 2022

- Virtual proxy:

- Annonymous access mode: no

- Authentication method: Header authentication dynamic user directory

- Header authentication header name: header_user

- Header authentication dynamic user directory: $ud\\$id

- has secure attribute (https): checked

- SameSite attribute (https): None

- Hosts white list: my QlikSense server domain (using just one QS server)

- I created cert/key/root files exported from the QMC/Certificates

- Certificates were converted to .pem using openssl. "client.pem" file contains both: a private key and a certificate

- I'm using this certificate sucesfully for a QPS API call

- Engine\Settings.ini configured with:

EnableTTL=1

SessionTTL=30

(and extra CR-LF line)

I'd like to perform a simple call (already tested via Engine API explorer) to method EvaluateEx in order it to perform a simple expression evaluation: =127

I receive three JSON messages, and then it gets stucked eternally waiting for a third message (never ends the while loop):

{'jsonrpc': '2.0', 'method': 'OnAuthenticationInformation', 'params': {'userId': 'xxx...', 'userDirectory': 'xxx...', 'logoutUri': 'https://(domain)/(proxy-prefix)/qps/user', 'serverNodeId': 'e13.....-....-....-....-.........b68', 'mustAuthenticate': False}} result JSON 1 {'jsonrpc': '2.0', 'method': 'OnConnected', 'params': {'qSessionState': 'SESSION_CREATED'}} result JSON 2 {'jsonrpc': '2.0', 'id': 1, 'error': {'code': -32602, 'parameter': 'Invalid handle', 'message': 'Invalid Params'}}

I'm using the next code:

QLIK_SENSE_SERVER_DOMAIN=... USER_DIRECTORY = ... USER_ID = ... # having Root Admin role # QlikSense Server: using app-id hash that I get from the app URL: # https://.../sense/app/6749d6...981c/overview APP_ID='6749d6..-....-....-...-........981c' PROXY_PREFIX = ... # Your virtual proxy prefix ENGINE_API_URL = f"wss://{QLIK_SENSE_SERVER_DOMAIN}/{PROXY_PREFIX}/app/{APP_ID}/" import websocket import json import ssl # QlikSense server trust this certificate, tested with another API: cert_path = 'cert_and_key.pem' # this file contains both, the private key and the certificate # Create a custom ssl context ssl_context = ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH) ssl_context.load_cert_chain(certfile=cert_path, keyfile=cert_path) header_user = { 'header_user': f'{USER_DIRECTORY}\{USER_ID}', 'X-Qlik-user': f'UserDirectory={USER_DIRECTORY};UserId={USER_ID}' } ws = websocket.create_connection(ENGINE_API_URL, sslopt={"cert_reqs": ssl.CERT_NONE, "ssl_context": ssl_context}, header=header_user) message={ "handle": 1, "method": "EvaluateEx", "params": { "qExpression": "=127" }, "id": 1, "outKey": -1, } ws.send(json.dumps(message)) result = ws.recv() print(json.loads(result)) i=1 while result: result=ws.recv() # print(f"result {i}", result) y = json.loads(result) if result is not None and result!="" else "None" print(f"result JSON {i}", y) i+=1 ws.close()

What means the Invalid handle / Invalid Params with error code = -32602?

I can't see any parameter apparently being invalid in the "message" variable.

I tried both: adding two extra params "id" and "outKey" and also not adding anything extra, yet the result is the same

Why does it get stucked? It seems like if the server wasn't ending the web socket properly

Re: Can't perform a simple Engine API call to an app using python

Øystein_Kolsrud — Fri, 04 Aug 2023 07:52:09 GMT

You need to open the app first. So before you call "EvaluateEx", send this message and wait for its response:

{ "jsonrpc": "2.0", "id": 1, "method": "OpenDoc", "handle": -1, "params": [ "<AppId>" ] }

That tells the engine to load the app into memory and create a session for you. After that you can run the method "EvaluateEx" given the handle that is returned by your OpenDoc call (which will almost always be 1).

If you'd like to learn more about the low level workings of the engine API, then this series of blog posts might be of interest to you:

https://community.qlik.com/t5/Design/Dissecting-the-Engine-API-Part-5-Multiple-Hypercube-Dimensions/ba-p/1841618

In particular, part two of that series discusses handles: https://community.qlik.com/t5/Design/Dissecting-the-Engine-API-Part-5-Multiple-Hypercube-Dimensions/ba-p/1841618

Re: Can't perform a simple Engine API call to an app using python

virilo_tejedor — Tue, 08 Aug 2023 10:16:50 GMT

Thanks a lot @Øystein_Kolsrud for helping me and for sharing your fabulous "Let's Dissect the Qlik Engine API" blog posts.

After reading it, I was able to have my first conversation with Engine API using websockets.

For those who it may help, this is the working code:

# -*- coding: utf-8 -*- """ Links (must read): "Qlik Sense: call Qlik Sense Engine API with Python" by Damien Villaret https://community.qlik.com/t5/Official-Support-Articles/Qlik-Sense-call-Qlik-Sense-Engine-API-with-Python/ta-p/1716089 "Let's Dissect the Qlik Engine API" by Øystein Kolsrud: 1. RPC Basics: https://community.qlik.com/t5/Qlik-Design-Blog/Let-s-Dissect-the-Qlik-Engine-API-Part-1-RPC-Basics/ba-p/1734116 2. Handles: https://community.qlik.com/t5/Qlik-Design-Blog/Let-s-Dissect-the-Qlik-Engine-API-Part-2-Handles/ba-p/1737186 3. Generic Objects: https://community.qlik.com/t5/Qlik-Design-Blog/Let-s-Dissect-the-Qlik-Engine-API-Part-3-Generic-Objects/ba-p/1761962 4. Hypercubes: https://community.qlik.com/t5/Qlik-Design-Blog/Let-s-Dissect-the-Qlik-Engine-API-Part-4-Hypercubes/ba-p/1778450 5. Multiple-Hypercube-Dimensions: https://community.qlik.com/t5/Design/Dissecting-the-Engine-API-Part-5-Multiple-Hypercube-Dimensions/ba-p/1841618 """ import websocket # pip install websocket-client import ssl import json ''' QLIK_GOBLAL_CONTEXT ia a special handle that identifies what is called the "Global" context which is a context that is associated with the Qlik Sense system it self See: https://community.qlik.com/t5/Design/Let-s-Dissect-the-Qlik-Engine-API-Part-2-Handles/ba-p/1737186 ''' QLIK_GOBLAL_CONTEXT = -1 # entry point for base functionality like "OpenDoc" QLIK_SENSE_SERVER_DOMAIN='myqlikserver.mycompany.com' USER_DIRECTORY = ... USER_ID = ... # Root Admin user # USER_ID needs permissions granted to access the app APP_ID APP_ID='6749d6..-....-....-...-........981c' PROXY_PREFIX = ... # Your virtual proxy prefix ENGINE_API_URL = f"wss://{QLIK_SENSE_SERVER_DOMAIN}/{PROXY_PREFIX}/app/{APP_ID}/" # Use pem format. In my case, both key and cert are stored in the same file cert_path = 'cert_and_key.pem' key_path= cert_path # Create a custom ssl context ssl_context = ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH) ssl_context.load_cert_chain(certfile=cert_path, keyfile=key_path) header_user = { 'header_user': f'{USER_DIRECTORY}\{USER_ID}', # you've to declare 'header_user' literal in the virtual-proxy setting called "header authentication header name" } ws = websocket.create_connection(ENGINE_API_URL, sslopt={"cert_reqs": ssl.CERT_NONE, "ssl_context": ssl_context}, header=header_user) msg_dummy_first = { "jsonrpc": "2.0", "id": 0, "method": "QTProduct", "handle": QLIK_GOBLAL_CONTEXT, "params": [] } msg_open_doc = { "jsonrpc": "2.0", "id": 1, "method": "OpenDoc", "handle": QLIK_GOBLAL_CONTEXT, "params": [ APP_ID ] } msg_evaluate_expression={ "handle": 1, "method": "EvaluateEx", "params": { "qExpression": "=127" }, "id": 2, "outKey": -1, } print('dummy call (this is necessary since, for some authentication methods, the "OnAuthenticationInformation" message will be sent by the server only after a valid message has been received from the client. )\n') ws.send(json.dumps(msg_dummy_first)) result = json.loads(ws.recv()) print(result) print() if 'params' in result and 'mustAuthenticate' in result['params']: if result['params']['mustAuthenticate']==False: print("correctly authenticated") else: print("authentication error:") else: print("WTF! no authentication info!!!") print("\nopening the doc:", APP_ID, "\n") ws.send(json.dumps(msg_open_doc)) open_doc_id=msg_open_doc['id'] result = json.loads(ws.recv()) print(result, "\n") i=1 doc_handle = False while result and not doc_handle: result=ws.recv() parsed_result = json.loads(result) if result is not None and result!="" else "None" print(f"result JSON {i}\n", parsed_result) i+=1 if parsed_result is not None and parsed_result.get('id', None)==open_doc_id: doc_handle=parsed_result.get('result',{}).get('qReturn', {}).get('qHandle', False) print(f"\nreceived handle {doc_handle} for app {APP_ID}\n") msg_evaluate_expression['handle'] = doc_handle print("simple call\n") ws.send(json.dumps(msg_evaluate_expression)) result = ws.recv() print(json.loads(result)) print() ws.close()