Deep dives into specific back-end technologies which allow for the extension of Qlik to fit the needs of the enterprise.
Last week, @Jesse_Paris covered (1) what are External Program Tasks, (2) how to create External Program Tasks, and (3) how to schedule External Program Tasks here: https://community.qlik.com/t5/Qlik-Architecture-Deep-Dive-Blog/External-program-tasks-in-Qlik-Sense/... His post covers how to schedule the External Program Task on a time-based schedule which solves one type of use case.
This week we'll cover how to chain the External Program Task to reload tasks and vice versa. With the capability to trigger an External Program Task before or after a reload task combined with the capability to trigger the External Program Tasks on a time interval, a site can fully integrate Qlik Sense into an end-to-end work-flow.
Before we start, a pre-requisite before going down this path will be to have some tool or scripted approach which can interact with the Qlik Sense Repository API. Postman is an easy tool to use and there's a helpful Qlik Support Article which will walk you through that process.
Reload Task > External Program Task
In this walk-through will we execute the external program task upon the successful completion of a reload task. An obvious use case for this would be to have a script that opens the app which just reloaded in order to warm the cache for end users.
We will start assuming that you have an External Program Task and a Reload Task. The high-level steps will be:
- GET the External Program Task details
- GET the Reload Task details
- (POST) CREATE a CompositeEventOperational
- (POST) CREATE a CompositeEvent (the link between the two tasks)
For our environment we will have the following response for the External Program Task (GET /qrs/externalprogramtask/{id}) :
{
"id": "b8c12920-858f-42a4-a799-741e46e45c73",
"createdDate": "2019-06-17T19:16:46.492Z",
"modifiedDate": "2019-06-17T19:16:46.492Z",
"modifiedByUserName": "INTERNAL\\sa_repository",
"customProperties": [],
"path": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
"parameters": " -ExecutionPolicy Bypass C:\\Temp\\preLoad.ps1",
"qlikUser": null,
"operational": {
"id": "7a0506ed-4fd6-4e4d-bd2d-32d6e2c81c2e",
"lastExecutionResult": {
"id": "2acfbd42-4e5e-48f1-bd4d-c85fe975360f",
"executingNodeName": "",
"status": 0,
"startTime": "1753-01-01T00:00:00Z",
"stopTime": "1753-01-01T00:00:00Z",
"duration": 0,
"fileReferenceID": "00000000-0000-0000-0000-000000000000",
"scriptLogAvailable": false,
"details": [],
"scriptLogLocation": "",
"scriptLogSize": -1,
"privileges": null
},
"nextExecution": "1753-01-01T00:00:00Z",
"privileges": null
},
"name": "CacheApp-5k_Random_Data",
"taskType": 1,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"tags": [],
"privileges": null,
"schemaPath": "ExternalProgramTask"
}
For the reload task in question we will have this response to a GET /qrs/reloadtask/{id} request:
[
{
"id": "5afd0e10-f9b5-4853-beb9-b84450059774",
"createdDate": "2019-06-17T19:16:17.518Z",
"modifiedDate": "2019-06-17T19:16:17.518Z",
"modifiedByUserName": "QMI-QS-CLN\\vagrant",
"customProperties": [],
"app": {
"id": "5cbfd02a-e0b1-4c75-802b-cbfefc369d32",
"name": "5k Random Data",
"appId": "",
"publishTime": "2019-06-17T15:41:02.887Z",
"published": true,
"stream": {
"id": "aaec8d41-5201-43ab-809f-3063750dfafd",
"name": "Everyone",
"privileges": null
},
"savedInProductVersion": "12.334.3",
"migrationHash": "98d482c3f964dccf69cbfb9a00e0c048ea6eb221",
"availabilityStatus": 0,
"privileges": null
},
"isManuallyTriggered": false,
"operational": {
"id": "f18c7ab9-0bab-4414-a559-b7e9516974a8",
"lastExecutionResult": {
"id": "8272d9ba-9278-4c9b-8fef-1dffff14d7b2",
"executingNodeName": "",
"status": 0,
"startTime": "1753-01-01T00:00:00Z",
"stopTime": "1753-01-01T00:00:00Z",
"duration": 0,
"fileReferenceID": "00000000-0000-0000-0000-000000000000",
"scriptLogAvailable": false,
"details": [],
"scriptLogLocation": "",
"scriptLogSize": -1,
"privileges": null
},
"nextExecution": "1753-01-01T00:00:00Z",
"privileges": null
},
"name": "Reload task of 5k Random Data",
"taskType": 0,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"tags": [],
"privileges": null,
"schemaPath": "ReloadTask"
}
]
With those stored for reference so that we can extract out elements we will create a CompositeEventOperational by sending the following API call: POST /qrs/compositeeventoperational, body: {}
In this walk-through the response is:
{
"id": "e7f14883-1bcd-4be3-9536-167f502ab7d3",
"createdDate": "2019-06-17T19:22:21.828Z",
"modifiedDate": "2019-06-17T19:22:21.828Z",
"modifiedByUserName": "INTERNAL\\sa_repository",
"timesTriggered": 0,
"privileges": null,
"schemaPath": "CompositeEventOperational"
}
The final step is to create the link between the two tasks. To do this we will issue a POST call to /qrs/compositeevent. For the body we will need elements from the three final calls. Our body is:
{
"timeConstraint":{
"seconds":0,
"minutes":360,
"hours":0,
"days":0
},
"compositeRules": [
{
"reloadTask": {
"id": "5afd0e10-f9b5-4853-beb9-b84450059774",
"name": "Reload task of 5k Random Data",
"taskType": 0,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"privileges": null
},
"schemaPath": "CompositeEvent.Rule"
}
],
"operational": {
"id": "e7f14883-1bcd-4be3-9536-167f502ab7d3",
"timesTriggered": 0,
"privileges": null
},
"name": "app-cache-trigger",
"enabled": true,
"eventType": 1,
"externalProgramTask": {
"id": "b8c12920-858f-42a4-a799-741e46e45c73",
"operational": null,
"name": "CacheApp-5k_Random_Data",
"taskType": 1,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"privileges": null
},
"privileges": null,
"schemaPath": "CompositeEvent"
}
The elements which will need to be adjusted to fit your environment are:
- compositeEvent > reloadTask > ID & Name
- Determinable from the
GET /qrs/reloadtaskcall - Notes:
- Insofar as you are chaining to a reload task then the above skeleton is correct. If chaining to another task type, then the reloadTask element will need to be renamed to the appropriate task type (userSyncTask,reloadTask)
- Additionally the taskType param will need to be adjusted to reflect the appropriate type of task. From our trusty
GET /qrs/about/openapi/mainwe can get the mapping:
- Determinable from the
"taskType": {
"type": "integer",
"enum": [
0,
1,
2
],
"x-enumNames": [
"Reload",
"ExternalProgram",
"UserSync"
]
- Operational > ID
- This is from the creation of the Operational the step before
- Name: The name of the trigger
- externalProgramTask > ID & Name
- From the
GET /qrs/externalprogramtask/fullcall earlier - Same notes as compositeEvent > reloadTask. If you adjust the task type before to rename the element and change to the appropriate taskType
- From the
Once we get the success response (a 201 HTTP response), our QMC looks as follows:
External Program Task > Reload Task
In this section will we do the reverse. The trigger for the reload task will be the successful execution of the External Program Task. This is the exact style of chain that we have used in our Telemetry Dashboard project. In it we are executing a series of Engine API calls to build out meta-data on all the apps which will then be consumed in the Qlik app which reloads after it. There are a litany of other use cases, but the key use-case here involves doing things outside of Qlik Sense prior to an application's reload.
We will start assuming that you have an External Program Task and a Reload Task. The high level steps will be:
- GET the External Program Task details
- GET the Reload Task details
- (POST) Update the Reload Task to link it to the External Program Task
For the External Program Task we have:
{
"id": "6c65ace8-0a81-4b4f-96d8-43573d32f9e0",
"createdDate": "2019-06-17T20:19:12.226Z",
"modifiedDate": "2019-06-17T20:19:12.226Z",
"modifiedByUserName": "INTERNAL\\sa_repository",
"customProperties": [],
"path": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
"parameters": " -ExecutionPolicy Bypass C:\\Temp\\preLoad.ps1",
"qlikUser": null,
"operational": {
"id": "4c70c20c-779e-4b07-942b-0f563cd33e66",
"lastExecutionResult": {
"id": "1174e76a-ded9-4de9-8ebe-11e732974a11",
"executingNodeName": "",
"status": 0,
"startTime": "1753-01-01T00:00:00Z",
"stopTime": "1753-01-01T00:00:00Z",
"duration": 0,
"fileReferenceID": "00000000-0000-0000-0000-000000000000",
"scriptLogAvailable": false,
"details": [],
"scriptLogLocation": "",
"scriptLogSize": -1,
"privileges": null
},
"nextExecution": "1753-01-01T00:00:00Z",
"privileges": null
},
"name": "CacheApp-5k_Random_Data",
"taskType": 1,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"tags": [],
"privileges": null,
"schemaPath": "ExternalProgramTask"
}
For the Reload Task we have:
{
"id": "13bbe435-c8da-408a-9a1d-3afebcc735da",
"createdDate": "2019-06-17T20:18:42.905Z",
"modifiedDate": "2019-06-17T20:18:42.905Z",
"modifiedByUserName": "QMI-QS-CLN\\vagrant",
"customProperties": [],
"app": {
"id": "5cbfd02a-e0b1-4c75-802b-cbfefc369d32",
"name": "5k Random Data",
"appId": "",
"publishTime": "2019-06-17T15:41:02.887Z",
"published": true,
"stream": {
"id": "aaec8d41-5201-43ab-809f-3063750dfafd",
"name": "Everyone",
"privileges": null
},
"savedInProductVersion": "12.334.3",
"migrationHash": "98d482c3f964dccf69cbfb9a00e0c048ea6eb221",
"availabilityStatus": 0,
"privileges": null
},
"isManuallyTriggered": false,
"operational": {
"id": "4c4eda49-7dd6-41ae-8b6a-f0bb3324a433",
"lastExecutionResult": {
"id": "13c75400-b8e1-41e4-b78b-d72df4bb5459",
"executingNodeName": "",
"status": 0,
"startTime": "1753-01-01T00:00:00Z",
"stopTime": "1753-01-01T00:00:00Z",
"duration": 0,
"fileReferenceID": "00000000-0000-0000-0000-000000000000",
"scriptLogAvailable": false,
"details": [],
"scriptLogLocation": "",
"scriptLogSize": -1,
"privileges": null
},
"nextExecution": "1753-01-01T00:00:00Z",
"privileges": null
},
"name": "Reload task of 5k Random Data",
"taskType": 0,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"tags": [],
"privileges": null,
"schemaPath": "ReloadTask"
}
With these pieces of information we will create the link by calling POST /qrs/reloadtask/update with a body in this vein:
{
"compositeEvents":[
{
"timeConstraint":{
"seconds":0,
"minutes":360,
"hours":0,
"days":0
},
"name":"appPreload",
"enabled":true,
"eventType":1,
"reloadTask":{
"id":"13bbe435-c8da-408a-9a1d-3afebcc735da"
},
"compositeRules":[
{
"ruleState":1,
"externalProgramTask":{
"id":"6c65ace8-0a81-4b4f-96d8-43573d32f9e0",
"name":"CacheApp-5k_Random_Data"
}
}
],
"privileges":[
"read",
"update",
"create",
"delete"
]
}
]
}
The elements which will need adjusted are:
- compositeEvent > name: This is the name of the trigger
- compositeEvent > reloadTask > ID: The ID of the reload task
- compositeEvent > compositeRules > externalProgramTask > id & Name: The Name and ID of the External Program task.
The resulting view in the QMC:
At this point you will have the capability to construct any variant of chain involving an External Program Task.