Qlik Community

Qlik Architecture Deep Dive Blog

Deep dives into specific back-end technologies which allow for the extension of Qlik to fit the needs of the enterprise.

Employee
Employee

External program tasks in Qlik Sense are simply another task type that exists, but not fully exposed by the QMC. They are tasks that are triggered from the QMC but allow for commands to be run in a command shell process behind the scenes; which in turns means you can run more or less anything you can run on the command prompt.

External program tasks in Qlik Sense are simply another task type that exists, but not fully exposed by the QMC. They are tasks that are triggered from the QMC but allow for commands to be run in a command shell process behind the scenes; which in turns means you can run more or less anything you can run on the command prompt. Once you define your command and create a new task, you can both trigger and chain these types of tasks in the QMC.

I will be using JavaScript for all the examples which means that node.js needs to be in your PATH variable (the location of node.exe). Again, you can run any command that you can run in the windows command prompt, so these examples are only for demo purposes.

Before we get started, we need to understand that IF you are calling scripts in your command, the script needs to be accessible by the user running the Qlik services. In this example, there’s a folder on the root C: drive called ‘externalTasksExample’, in this folder there is a createFile.js file which simply creates a new file in the same folder.

The file looks like:

var fs = require('fs');
var outputText = "This text will be in my output file.";
const data = new Uint8Array(Buffer.from(outputText));
fs.writeFile('C:\\externalTasksExample\\testCreateFile.txt', data, function (err) {
  if (err) throw err;
  console.log('The file has been saved!');
});

You can run the file from the command prompt by doing the following:

C:\externalTasksExample>“C:\Program Files\Qlik\Sense\ServiceDispatcher\Node\node.exe” createFile.js

When the command runs successfully, you should have a new file created in the externalTasksExample folder called “testCreateFile.txt” with the contents: “This text will be in my output file.”

Now we will delete this testCreateFile.txt and try to run that same JavaScript file using an external program task in Qlik Sense. The first thing we need to do is actually create the task.

The endpoint is:

POST /qrs/externalprogramtask

With the body being:

{
  "path": "C:\\Program Files\\Qlik\\Sense\\ServiceDispatcher\\Node\\node.exe",
  "parameters": "C:\\externalTasksExample\\index.js",
  "name": "ExampleExternalTask",
  "taskType": 1,
  "enabled": true,
  "taskSessionTimeout": 1440,
  "maxRetries": 0,
  "privileges": null,
  "schemaPath": "ExternalProgramTask"
}

Once you post this into Qlik, you should be able to go to the QMC and see the following task in the task list.

You should be able to select the task, click 'Start', and the same file should appear in 'C:\externalTasksExample'. You are now running an external task. Remember you don't have to use node.js, you can trigger anything you want as long as it can be run from the command line.

Scheduling

When it comes to scheduling an external program task, the easiest way is to create a "scheduled trigger" for a dummy reload task in the QMC, then move the trigger from the reload task to the external program task. You will need the ID of the external program task for this, which you would have gotten back in a response from the POST in the previous section, or you can use the GET call below to get a list of all external program tasks and their IDs.

GET /qrs/externalprogramtask

To start, go back to the Tasks list in the QMC and create a new task. Give it a memorable name (I will use 'tempTask'), and choose any application. Then under 'Triggers -> Actions', 'Create new scheduled trigger'. Fill out the scheduled trigger as if it was for the external program task.

Hit ok, click apply on the reload task, and you will then be taken to the 'Tasks' screen in the QMC. Next, we will use the API to find the new scheduled trigger we created.

GET /qrs/schemaevent?filter=name eq 'testTrigger'

Note the ID returned for this object, then get:

GET /qrs/schemaevent/{ID}

You will want to copy the entire response, which should look like something below.

{
    "id": "23268794-35f3-42f6-a9c0-57d91e2040cd",
    "createdDate": "2019-02-16T01:42:38.583Z",
    "modifiedDate": "2019-02-16T01:42:38.583Z",
    "modifiedByUserName": "DESKTOP-GS8LAA5\\jesse paris",
    "timeZone": "America/Chicago",
    "daylightSavingTime": 0,
    "startDate": "2019-02-15T19:47:30.000",
    "expirationDate": "9999-01-01T00:00:00.000",
    "schemaFilterDescription": [
        "* * - * * * * *"
    ],
    "incrementDescription": "0 0 0 0",
    "incrementOption": 0,
    "operational": {
        "id": "f113579d-3c68-4313-b448-1d934f09c998",
        "nextExecution": "2019-02-16T01:47:30.000Z",
        "timesTriggered": 0,
        "privileges": null
    },
    "name": "testTrigger",
    "enabled": true,
    "eventType": 0,
    "reloadTask": {
        "id": "4eb7f372-1333-4f1c-842d-678e6e3b3acb",
        "operational": {
            "id": "fe6cc117-5fcb-44bb-a18d-1c35dade5189",
            "lastExecutionResult": {
                "id": "9f4b69d7-c97e-4262-ab52-07401b93df7e",
                "executingNodeName": "desktop-gs8laa5",
                "status": 7,
                "startTime": "2019-02-16T00:53:17.140Z",
                "stopTime": "2019-02-16T00:53:19.171Z",
                "duration": 2031,
                "fileReferenceID": "792555fa-e96a-41ea-afbe-59b0028d8439",
                "scriptLogAvailable": false,
                "details": [
                    {
                        "id": "dc3a8eb0-9d44-4b32-aac7-bafb9c7d4e32",
                        "detailsType": 2,
                        "message": "Changing task state to Triggered",
                        "detailCreatedDate": "2019-02-16T00:53:17.140Z",
                        "privileges": null
                    },
                    {
                        "id": "8551a721-7d48-45fe-8505-c2fe6d1a1ee1",
                        "detailsType": 2,
                        "message": "Trying to start task. Sending task to slave scheduler desktop-gs8laa5",
                        "detailCreatedDate": "2019-02-16T00:53:17.468Z",
                        "privileges": null
                    },
                    {
                        "id": "55e9458f-af57-4f4b-92c5-463c9ad5ae67",
                        "detailsType": 2,
                        "message": "Changing task state from Triggered to Started",
                        "detailCreatedDate": "2019-02-16T00:53:17.718Z",
                        "privileges": null
                    },
                    {
                        "id": "5836b6c3-2287-4c8f-8c9b-af39dbea4176",
                        "detailsType": 2,
                        "message": "Reference to scriptlog added",
                        "detailCreatedDate": "2019-02-16T00:53:19.093Z",
                        "privileges": null
                    },
                    {
                        "id": "d0c66f1d-3238-43f5-a7c1-358a8d40a746",
                        "detailsType": 2,
                        "message": "Changing task state from Started to FinishedSuccess",
                        "detailCreatedDate": "2019-02-16T00:53:19.233Z",
                        "privileges": null
                    }
                ],
                "privileges": null
            },
            "nextExecution": "2019-02-16T01:47:30.000Z",
            "privileges": null
        },
        "name": "TestTaskDeleteME",
        "taskType": 0,
        "enabled": true,
        "taskSessionTimeout": 1440,
        "maxRetries": 0,
        "privileges": null
    },
    "userSyncTask": null,
    "externalProgramTask": null,
    "privileges": null,
    "schemaPath": "SchemaEvent"
}

We will take this response, remove the 'reloadTask' object, and add a reference to our external program task. Here we will need the ID that was returned when we created a new external program task. A final body should look like the following:

{
    "id": "23268794-35f3-42f6-a9c0-57d91e2040cd",
    "createdDate": "2019-02-16T01:42:38.583Z",
    "modifiedDate": "2019-02-16T01:42:38.583Z",
    "modifiedByUserName": "DESKTOP-GS8LAA5\\jesse paris",
    "timeZone": "America/Chicago",
    "daylightSavingTime": 0,
    "startDate": "2019-02-15T19:47:30.000",
    "expirationDate": "9999-01-01T00:00:00.000",
    "schemaFilterDescription": [
        "* * - * * * * *"
    ],
    "incrementDescription": "0 0 0 0",
    "incrementOption": 0,
    "operational": {
        "id": "f113579d-3c68-4313-b448-1d934f09c998",
        "nextExecution": "2019-02-16T01:47:30.000Z",
        "timesTriggered": 0,
        "privileges": null
    },
    "name": "testTrigger",
    "enabled": true,
    "eventType": 0,
    "reloadTask": null,
    "userSyncTask": null,
    "externalProgramTask": {
        "id": "e7ba25a8-a56c-4529-bc9a-af41755d3feb"
    },
    "privileges": null,
    "schemaPath": "SchemaEvent"
}

Using this body, you will update the schema event in the Qlik Sense Repository service with

PUT /qrs/schemaevent/{ID}

Once this is done, the task you created will now be assigned to the external program task. If you go back to the temporary reload task you created, you will notice that it no longer has a scheduled trigger associated to it, and you can delete that task. Any changed to the trigger need to be made to the schema event through the API, and any changes to the task itself needs to be made to the external program task through the API.

Happy APIing!

8 Comments
Contributor III
Contributor III

Hi Jess,

I am trying to implement Qliksense external tasks. when I am trying to run the task in QMC , it is failing 

here is my command in CMD : C:\qsense\Utilities\Notifier\QS.Notifier.exe "MISFee" and it is working fine. when create external task my body is 

{
"path": "C:\\qsense\\Utilities\\Notifier\\QS.Notifier.exe",
"parameters": "QS.Notifier.exe MISFee",
"name": "ExternalTaskTest",
"taskType": 1,
"enabled": true,
"taskSessionTimeout": 1440,
"maxRetries": 0,
"privileges": null,
"impactSecurityAccess": false,
"schemaPath": "ExternalProgramTask"
}

 

task is created in QMC but failing to run, I think my path, parameters are wrong. could you please help on this.

 

 

 

0 Likes
1,654 Views
Employee
Employee

Your path is the path to your application, then you are passing the same value in the parameters. If you want MISFree to be a param to QS.Notifer.exe, then just have the param be "MISFee".

0 Likes
1,612 Views
Contributor III
Contributor III

Hi Jesse,

Thank you.  Changing the parameter worked and this QS.Notifer.exe should be on publisher node not engine node. thanks for you help. 

 

Would it possible to do this task creation and scheduling through Qliksense app using rest connector like triggering NPrinting task from Qliksense so that it would be easy for Qliksense developer/ admins to create these type of task directly from Qliksense instead of using Postman. 

Cheers,

RK

Cheers,

Raghav.

0 Likes
1,592 Views
Employee
Employee

Yes, you could do that. Then obviously schedule those 'reloads' as well.

0 Likes
1,578 Views
Contributor III
Contributor III

Thank you. Can you please provide me guidance for connecting QRS using Qlik rest connector in QS. 

 

Cheers,

Raghav 

0 Likes
1,574 Views
Contributor
Contributor
0 Likes
689 Views