Set up a text analytics sentiment analysis application (including example qvf)
This quick how to guide will help you understand how to setup a sentiment analysis application for any text strings you like; tweets, Facebook posts and comments etc. I've also included an example application for you to work with to see how the code works. Follow the simple steps below to get something working to trial it out on your data.
The diagram below shows the simple flow required in this example. It assumes that there are multiple languages in the text strings you wish to analyze as this is a complication that needs to be considered. I have used the two services in question as they met my specific needs. AYLIEN was my preferred language recognition API after some research and testing. REPUSTATE was my preferred sentiment analyser as this seems to cope with Arabic better than other services, AYLIEN offer a sentiment engine too so you can keep it all with the same provider.
STEP 1 - Sign up for accounts for two API services
You will need to sign up to these accounts. There are demo accounts available for both of these services which I have used for this example. You may need to explore the commercial arrangements with these providers depending on your own use cases.
if you have an account, sign in. If not click on the sign up button and enter your details. You will receive an email with a link to validate your account so ensure you have access to your email.
Once you have signed in you will see a screen as below:
you will need the API Key for use with the Web Connector
STEP 2 - Understand the URL Encoding required
The text you need to send to the APIs needs to be encoded so it can be sent as a URL, we do this using a sub routine in the script.
This sub routine will need to be amended/augmented when you deal with languages or characters that are not yet covered, for example when dealing with languages which do not use standard latin alphabet characters (such as Arabic, Chinese etc). In this example I have added encoding to cater for Arabic characters and many different unicode characters and symbols but this may not be complete.
The URL Encoding can be found in the second section of the script in the example application attached.
For more information on URL encoding and the following loop script approach see the Qlik help site.
If you need to add any characters into the list simply add an addition let statement copying those you can see here.
STEP 3 - Enter your own test data as an inline table or load your own file
The 3rd section of the script holds some test data. Just two records in different languages in my example. replace this with your own by replacing the data in the inline table or by loading in the data from any source as you would normally.
PLEASE NOTE: I have labelled the field I wish to analyse 'Comment'. If you wish to change this you will need to make a change in the Sentiment section of the script.
STEP 4 - Enter your own AYLIEN API Information you setup in Step 1.
The AYLIEN API settings can be entered in the script as below. There are two values that need to be entered as covered in step 1.
STEP 5 - Enter your own REPUSTATE API Information you setup in Step 1.
The REPUSTATE connector needs to be setup through the Qlik Web Connectors interface. I am using the November 2017 release of the Web Connectors and it is worth commenting that these connectors may change.
I am using the Qlik Repustate Connector which can be found at the link below (you may have a different host name).
We only need to setup the connector to save the API key so you do not need to copy any script as it is already setup with variables in the example application.
Select the Sentiment table and click on Parameters
Enter the API Key, some dummy text (do not worry what this is as we will not be copying the URL)
Press 'Save Inputs and Run Table'
STEP 6 - Now you're ready to reload your application.
The Sentiment section of the script runs through a loop of each text value. It will pass through the AYLIEN language recognition API to identify the language which is a required parameter for the REPUSTATE sentiment API.
Points to Consider:
This approach uses an inline script model which passes one record at a time. It's quick and ideally suited to incremental loads where only the new records are passed through this logic.
There are bulk API options with REPUSTATE (and other services offering this) which may be better suited for large volume use cases (such as initial loads).
This is an example, the approach can be tailored to your needs and use whatever API services you prefer but the flow of how it is done is likely to be the same.