How to: sentiment analysis in Qlik Application Automation

Emile_Koslowski · Apr 6, 2022 6:13:29 AM

Sentiment analysis is an NLP machine learning task that analyzes a provided text and returns the detected sentiment.

In this article, we'll build an automation that analyzes the sentiment of support incidents from ServiceNow. The NLP machine learning platform used in this article is Expert.ai but feel free to use another NLP platform like Hugging Face.

The predicted sentiment will then be written back to ServiceNow and also to a MySQL database. The database is then used to feed a Qlik Sense app that analyses your support tickets.

See these articles if you're interested in other machine learning capabilities in Qlik Application Automation:

Environment

Qlik Application Automation
Qlik Sense - to further analyze support ticket data
Expert.ai - for sentiment analysis
ServiceNow - as a source for support tickets (Incidents)
MySQL database - to feed support ticket data (inc. sentiment) to the Qlik Sense app

The following image provides an overview of how these systems are tied together by the automation:

Prerequisites

Before the automation can be built, a few steps need to be undertaken to assure that all systems can work together. Most machine learning tasks take a lot of preparation, mostly to create a model. But in this scenario, we're only using NLP which uses pre-trained models.

1. Prepare support portal

Not every support system contains fields to store sentiment information. If you plan on writing this information back to your support system, add these fields to the ticket or incident object.

2. MySQL database

Make sure to create a MySQL database (or any other database) to store the predicted sentiment information together with the support incidents. This database will be used to import records to your Qlik Sense app. You can use a different type of database.

3. Qlik Sense App

Build a new Qlik Sense App that you'll use to analyze the support tickets. Make sure to feed the app with data from the previous step.

Automation building steps

The following steps will guide you to build an automation on support incident/ticket sentiment analysis. A full version of this automation can be found at the end of this article as 'Sentiment analysis automation example.json'. How to import and export automations

Go to your Qlik Sense tenant and create a new automation.
Search for the ServiceNow connector and drag the List New And Updated Incidents Incrementally block in the automation. When this block is executed for the first time, it will retrieve all incident records from ServiceNow. It will also set a pointer with the DateTime of its last execution. On the next executions, the block will only retrieve incidents that were created or updated after the pointer and it will update the pointer. Go to the block's Connection tab and connect your ServiceNow account.
Search for the Expert.ai connector and drag the Run Analysis Task block inside the loop. Go to the block's Connection tab and connect your Expert.ai account. Then go to the Inputs tab to further configure the block. Set Analyze Type to "Sentiment analysis" and set the language to "en" for English. Feel free to configure this to another language depending on your support portal. Finally, configure the Document Text parameter to the Description key of the current item in the loop from the List New And updated Incidents Incrementally block.
The next step is to write the predicted sentiment back to ServiceNow. In this example, we'll make a distinction between positive and negative sentiment. Add a Condition block to the automation and configure the overall sentiment (returned by Expert.ai) as the first argument and compare it to a threshold of your choosing. Note: the threshold needs to be a number between -100 (negative) and 100 (positive).
Now we can add the blocks to write back to ServiceNow. Search for the Update Incident block in the ServiceNow connector and add it to the automation twice. (One block for every outcome of the Condition block). For this example, we've added 2 custom fields to the Incident object in ServiceNow, one to store the overall sentiment and one boolean (Yes/No) that indicates if the incident has a sentiment lower than our threshold. More information on custom fields in ServiceNow can be found here.
As a next step, you could include alerting functionality to make your support team aware of the ticket with a negative sentiment below a certain threshold. In this example, we'll send an alert to a channel in Microsoft Teams. Search for the Microsoft Teams connector and add the Send Message block in the 'Yes' part of the condition block that validates the overall sentiment.
Add a block to write this information to a database and configure a reload for an app that imports this database. In this example, the Upsert Record block from the MySQL connector is used but feel free to change this to any other cloud storage platform. Add the block after the Condition block so it runs for every new and updated incident.
Finally, we'll trigger an app reload by adding the Do Reload block. Make sure that you add this block outside the loop created by the List New And Updated Incidents Incrementally block, otherwise it would get executed for every record returned from ServiceNow. Add it together with a Condition block to make sure that it doesn't get executed when no records are retrieved from ServiceNow.

Automation run mode

Since this automation only processes new and updated records support incidents, it's best to configure its run mode to Scheduled to make sure the automation is executed every x minutes. In this example, we've used 15 minutes but this will depend on your use case and type of customers.

Attached to this article, you'll find an exported version of the above automation as 'Sentiment analysis automation.json' See the How to import and export automations article to learn how to import exported automations.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.