Engine API - Extracting info beyond handle 1 with Python script

[Solvica]

Dear all,

We have successfully connected to the Engine API with a Python script (with the instructions discussed here). Now we are trying to extract the the data we need (for instance, an app´s load script), but we are not being able to go beyond handle number 1 (i.e., opening the app).

Once the websocket connection is established as the variable ws, I run the following:

ws.send(json.dumps({
	"handle": -1,
	"method": "GetDocList",
	"params": [],
	"outKey": -1,
	"id": 1
}))

ws.send(json.dumps({
	"method": "OpenDoc",
	"handle": -1,
	"params": [
		"{some app id}"
	],
	"outKey": -1,
	"id": 2
}))
ws.send(json.dumps({
	"handle": 1,
	"method": "GetScript",
	"params": {},
	"outKey": -1,
	"id": 3
}))

result = ws.recv()

while result:
    result=ws.recv()
    y = json.loads(result)
    print(y)

ws.close()

All the json bodies are directly copied from the queries done by the Engine API Explorer.

I get the expected results from the two first requests, but for request id 3 (method GetScript) I get the following error:

{'jsonrpc': '2.0', 'id': 3, 'error': {'code': -32602, 'parameter': 'Invalid handle', 'message': 'Invalid Params'}}

I am confused because the handle is correct and, although the message says that I have introduced "Invalid Params", the API guidance on this method (and others I have tried) says that there are no parameters to be introduced.

I have also searched the documentation for the python´s module you recommend to connect to the API but there is nothing about json bodies in the request message.

Am I missing any configuration or parameter either in Qlik or in my Python script?

Thanks a million for your guidance!

Ginés - Solvica

[Øystein_Kolsrud · Accepted solution]

It's hard to say what goes wrong here without seeing the responses you get from the engine. Are you sure the app opened correctly as expected? Because based on that error message, it sounds to me like there was no handle returned by that "OpenDoc" call.

I have written a couple of blog posts on the engine API that might be of interest to you. It attempts to explain fundamental concepts such as handles and request IDs which might be of use to you to understand what is going on in your case:

https://community.qlik.com/t5/Design/Dissecting-the-Engine-API-Part-5-Multiple-Hypercube-Dimensions/...

In particular, this post describes the concept of handles:

https://community.qlik.com/t5/Design/Let-s-Dissect-the-Qlik-Engine-API-Part-2-Handles/ba-p/1737186

[Solvica]

Hi again @Øystein_Kolsrud and thanks in advance for your support. 

I am adding now the full response (I have only avoided details like AppIDs and other app properties, but otherwise this is what I get). Does this help to understand better what might be going wrong?

++Sent raw: b"\x81\xcb\x1d\xeaS\xe3f\xc8;\x82s\x8e?\x86?\xd0s\xce,\xc6s\xc1p\x8f'\x8br\x8eq\xd9=\xc8\x14\x86i\xae<\x80Q\x83 \x97?\xc6s\xc1m\x8b!\x82p\x99q\xd9=\xb1\x0e\xcf=\xc8<\x96i\xa16\x9a?\xd0s\xce,\xc6s\xc1t\x8eq\xd9=\xdb."

++Sent decoded: fin=1 opcode=1 data=b'{"handle": -1, "method": "GetDocList", "params": [], "outKey": -1, "id": 1}'

++Sent raw: b'\x81\xee4\xc1\xee\x0eO\xe3\x83k@\xa9\x81j\x16\xfb\xce,{\xb1\x8b`p\xae\x8d,\x18\xe1\xccfU\xaf\x8abQ\xe3\xd4.\x19\xf0\xc2.\x16\xb1\x8f|U\xac\x9d,\x0e\xe1\xb5,P\xf1\x8dj\r\xa3\xdc:\x19\xa5\xd9<V\xec\xda9\x02\xf8\xc36Q\xa2\x8b#V\xf0\xdb:U\xf1\xdbk\x07\xa7\xdf=\x16\x9c\xc2.\x16\xae\x9bz\x7f\xa4\x97,\x0e\xe1\xc3?\x18\xe1\xccgP\xe3\xd4.\x06\xbc'

++Sent decoded: fin=1 opcode=1 data=b'{"method": "OpenDoc", "handle": -1, "params": ["some app ID"], "outKey": -1, "id": 2}'

++Sent raw: b'\x81\xc9E\x9fV\x01>\xbd>`+\xfb:dg\xa5v0i\xbftl \xeb>n!\xbdl!g\xd83u\x16\xfc$h5\xebt-e\xbd&`7\xfe;rg\xa5vz8\xb3v#*\xea"J \xe6t;e\xb2g-e\xbd?eg\xa5v28'

++Sent decoded: fin=1 opcode=1 data=b'{"handle": 1, "method": "GetScript", "params": {}, "outKey": -1, "id": 3}'


++Rcv raw: b'\x81U{"jsonrpc":"2.0","method":"OnConnected","params":{"qSessionState":"SESSION_CREATED"}}'

++Rcv decoded: fin=1 opcode=1 data=b'{"jsonrpc":"2.0","method":"OnConnected","params":{"qSessionState":"SESSION_CREATED"}}'

++Rcv raw: b'\x81h{"jsonrpc":"2.0","id":3,"error":{"code":-32602,"parameter":"Invalid handle","message":"Invalid Params"}}'

++Rcv decoded: fin=1 opcode=1 data=b'{"jsonrpc":"2.0","id":3,"error":{"code":-32602,"parameter":"Invalid handle","message":"Invalid Params"}}'


{'jsonrpc': '2.0', 'id': 3, 'error': {'code': -32602, 'parameter': 'Invalid handle', 'message': 'Invalid Params'}}


++Rcv raw: b'\x81\x7f\x00\x00\x00\x00\x00\x01t3{"jsonrpc":"2.0","id":1,"result":{"qDocList":[some very long app list with its properties]}}'

++Rcv raw: b'\x81~\x00\x8a{"jsonrpc":"2.0","id":2,"result":{"qReturn":{"qType":"Doc","qHandle":1,"qGenericId":"some app ID"}},"change":[1]}'

++Rcv decoded: fin=1 opcode=1 data=b'{"jsonrpc":"2.0","id":2,"result":{"qReturn":{"qType":"Doc","qHandle":1,"qGenericId":"some app ID"}},"change":[1]}'

{'jsonrpc': '2.0', 'id': 1, 'result': {'qDocList':[some very long app list with its properties]}}

{'jsonrpc': '2.0', 'id': 2, 'result': {'qReturn': {'qType': 'Doc', 'qHandle': 1, 'qGenericId': 'some app ID'}}, 'change': [1]}

Thank you also for the links to the blog posts, they will continue being of help in this trip to the core of the Engine 🙂

Gines

[Aiham_Azmeh · Accepted solution]

Hi @Solvica ,

It looks to me that you are sending the GetScript call before the engine manages to open the app (see OnConnected)

++Sent decoded: fin=1 opcode=1 data=b'{"method": "OpenDoc", "handle": -1, "params": ["some app ID"], "outKey": -1, "id": 2}'

++Sent raw: b'\x81\xc9E\x9fV\x01>\xbd>`+\xfb:dg\xa5v0i\xbftl \xeb>n!\xbdl!g\xd83u\x16\xfc$h5\xebt-e\xbd&`7\xfe;rg\xa5vz8\xb3v#*\xea"J \xe6t;e\xb2g-e\xbd?eg\xa5v28'

++Sent decoded: fin=1 opcode=1 data=b'{"handle": 1, "method": "GetScript", "params": {}, "outKey": -1, "id": 3}'


++Rcv raw: b'\x81U{"jsonrpc":"2.0","method":"OnConnected","params":{"qSessionState":"SESSION_CREATED"}}'

++Rcv decoded: fin=1 opcode=1 data=b'{"jsonrpc":"2.0","method":"OnConnected","params":{"qSessionState":"SESSION_CREATED"}}'

I am not familiar with the code examples in that community blog, but I guess you need to sequence you calls to the backend synchronously.

I also recommend you check our official python SDKs https://pypi.org/project/qlik-sdk it s build to run against Qlik Cloud, ...but you should be able to do raw authenticated RPC calls with the JWT against QS on windows

[Solvica]

Hi @Aiham_Azmeh, 

You were 100% right, the problem was that the requests to Handle 1 were being sent too soon, before the Engine could open the App.

As a reference for others that might come to see this post in the future when trying to solve a similar problem: I have simply added a pause in my script of 10 seconds after requesting the method "OpenDoc". Now it works fine.

Thank you @Aiham_Azmeh 

Ginés