Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Jan 31, 2024 9:06:12 AM
Oct 28, 2021 9:32:19 PM
In part one of this series, you looked at preparing Slack to allow a developer to use its APIs, and in part two, you gathered the data that is needed to send messages using the Slack API. In this final part, you will create a Job that allows you to send messages to users and channels in Slack.
You will need to identify which posts should be sent to your intended users, and this may require you to adjust the source data components to suit the data you wish to send messages about. Here is a high level overview of the process Talend uses, which may be different from your use case.
Content:
There are two parts of the process:
- The first involves collecting the details of users and channels that want to be notified, and what Community boards and/or topics (known as labels) they want to be informed about. This information is kept in a Snowflake database. Each user/channel is linked to the boards/topics they want to be notified about, and is also linked to a "live" status that is a Boolean. This allows us to switch notifications off for the user/channel temporarily, without removing the data.
- The second part involves collecting all of the Community post and comment data from Salesforce and storing it in Snowflake every 30 minutes. We store the data outside of Salesforce because it is faster to query the data in Snowflake; we are not limited to CPU cycles or how we can join the data, meaning that the SQL we use is simpler and significantly faster than if we tried to query directly in Salesforce. Our Slack_Data table (discussed in part 2) and user/channel configuration data are also stored in Snowflake.
Slack message format
There are two things you need to do to send Slack messages using Talend: build a Job, and decide how you want your messages to look within Slack. To make the latter as easy as possible, use Slack's Block Kit Builder. It allows you to build the look and feel you want, then gives you the code to create that look and feel in Slack.
As an example, here is the look and feel I put together for the Community unanswered post notifications. It is very basic and has just a title, date, and a series of links to posts that currently need answers.
You can achieve the required look and feel by dragging and dropping fields from the left side bar onto the display screen, which are converted to code on the right. You can then modify that code (if you choose) to tweak what you see in the display screen. I identified how the code could be dynamically built using Java String manipulation, then built the Job to accommodate this. I will show this code when I describe building the Job.
Another useful tool to help with Java String manipulation is Freeformatter.com. This helps you to "escape" all of the characters in the JSON code that do not work well with Java String manipulation. Since the Block Kit Builder produces JSON, you will find this tool very useful if you want to preserve formatting (such as quotes, new lines, tabs).
Job to send Slack messages
This Job can be run on its own, or as a child Job being called by a parent Job. In the case of Talend Community, our parent Job identifies users and channels that have posts to be informed about. The user/channel name and ID are passed to the child Job, then that Job sends out the messages associated with that user/channel. Below is a screenshot of the child Job that sends out the messages.
As in part 2, I will follow the numbering in explaining what each component does, except for the tLogRow components that are just used for logging to the System.out. These can be disabled by right-clicking the component and selecting Deactivate.
This Job requires some context variables to be able to function, shown below. These variables are used to pass values into this Job when it is called as a child Job, but they can also be used to run the Job without a parent Job.
- The first is SlackOAuthAccessToken. Set this to the token you created in part 1.
- Next is Username, the name of the Slack user or Slack channel the message needs to be sent to in this run of the Job.
- This isn’t as important for sending the messages (the ChannelId is important for that), but I have used it to search for relevant posts for our user/channel to be informed about. It’s a key for the data to be sent. You may find a better key to find the content you want to send to the user/channel.
- Last is ChannelId, the ID that the message will be sent to. If this is incorrect or not provided, the Job will fail.
Once these context variables are set, you can start building the Job.
| 1 | tJava | This component creates the message header. Since a user/channel might get several messages, the header message sets the subject “Talend Community Unanswered Posts” and includes the date. An example appears below. The channel has been set to xxxxxxxxxx and the date has been set to 2021-06-01.
{"channel":"xxxxxxxxxx",
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "Talend Community Unanswered Posts",
"emoji": true
}
},
{
"type": "section",
"text": {
"type": "mrkdwn"
"text": "*Date:* 2021-06-01"
}
},
{
"type": "divider"
}
]
}
The code used to create the JSON above, which was designed using Slack’s Block Kit Builder, is below. This is what is used in the tJava component.
String openingPost =
"{\"channel\":\""+context.ChannelId+"\",\r\n\"blocks\":
[\r\n\t\t{\r\n\t\t\t\"type\": \"header\",\r\n\t\t\t\"text\":
{\r\n\t\t\t\t\"type\": \"plain_text\",\r\n\t\t\t\t\"text\": \"Talend
Community Unanswered Posts\",\r\n\t\t\t\t\"emoji\":
true\r\n\t\t\t}\r\n\t\t},\r\n\t\t{\r\n\t\t\t\"type\":
\"section\",\r\n\t\t\t\"text\": {\r\n\t\t\t\t\"type\":
\"mrkdwn\",\r\n\t\t\t\t\"text\": \"*Date:*
"+routines.TalendDate.formatDate("yyyy-MM-dd",
routines.TalendDate.getCurrentDate())+"\"\r\n\t\t\t}\r\n\t\t},\r\n\t\t{\r
\n\t\t\t\"type\": \"divider\"\r\n\t\t}\r\n]\r\n}";
globalMap.put("openingPost", openingPost);
Note that the JSON String is set in the globalMap HashMap using the key "openingPost". |
| 2 | tFixedFlowInput | This component is used to pass the JSON created above to the tRestClient component used to call the API. A quick way of configuring this component is to connect it straight to the tRestClient. This will autocreate its schema. You can drop a tLogRow between the components if you wish. Once the schema is created, you need to add this to the string column:
((String)globalMap.get("openingPost"))
 |
| 4, 21 | tRestClient | tRestClient components are used in two areas. These components are configured in the same way and are used to send Slack messages using the Slack API, just at different times. The messages that they send are prepared by the components preceding them and are sent in the form of JSON Strings to the string input column. The Basic settings are shown below:  The URL has been hardcoded above. This serves its purpose here, but isn’t necessarily future proof - you may want to use a context variable that can easily be changed. But for ease of copying, the URL is: “https://slack.com/api/chat.postMessage”The HTTP Method is set to POST, and the Content Type and Accept Type are set to JSON. Everything else on the Basic Settings tab is left as the default. The Advanced Settings are shown below:  You need to configure the HTTP Headers on this tab. As is the case with all of the tRestClient components used for interacting with the Slack API, you need to set the “Content-type” to “application/json” and the “Authorization” to: "Bearer "+content.SlackOAuthAccessTokenThis is assuming that your token is being stored in a context variable called SlackOAuthAccessToken. Both of these components have tLogRow components after them, to send out the success or failure messages to the System.out. You can add logging here if you wish. |
| 7 | tDBInput (Snowflake) | This component is used to read in the data that needs to be sent to the user/channel (it is where the data you want to send a Slack message about is provided). The fields that are being returned to the Job are:
Your requirements may differ significantly from those being demonstrated, so this component could be any input component. |
| 8 | tSortRow | This component is used to sort the data by CreatedDate so that the newest posts are first. This could have been done using the DB component preceding it; it would be more efficient to add the sorting to the SQL query. If you decide to use an input component that is not sortable, the configuration of this component is shown below:  |
| 10 | tMap | This component is used to filter the data into sets of 10 records grouped by board and topic, because we only want our users/channels to be informed about the 10 most recent posts per board and topic combination that they have picked. The configuration of the component is shown below:  Pay attention to the filter in the green box. This only outputs records where the computed counter variable is 10 or lower. The expression to compute the counter variable is shown below: Numeric.sequence(row24.BOARD+"|"+row24.TOPIC,1,1)Here a new sequence is generated for every unique board/topic combination. If it already exists, it is incremented by 1. This allows you to count the number of board/topic pairs of records. You can then guarantee that you will not be notifying the users/channels of more than 10 records per board/topic pair. |
| 12 | tHashOutput | This component is used to store the records that you have just collected and sorted using the tSortRow and tMap. The records are stored in memory to be used in the next subJob. The tHashOutput needs no extra configuration, it simply needs to be connected to the tMap output. |
| 13, 16 | tHashInput | Component 13 is used to read the data collected by component 12 into the next SubJob. This component configuration is actually used twice, so pay attention when dealing with component 16. In order to connect this component to the tHashOutput that collects the data (component 12), you need to select the name of the component from the Component list drop-down. It is shown below in the red box.  You also need to set the schema, which can be copied from component 12. To set the schema, click the Edit schema button (in the blue box) and you will see the popup below:  Once the schema of component 12 and this component match, you are ready to move to the next component. |
| 14 | tAggregateRow | This component is used to achieve a bit of logic from the tHashInput data that is quite useful, but not well known. You can reuse the tHashInput data again and again. The tHashInput is being used to read in the data sorted and filtered in the previous subJob. What you want to do here is process the data in board/topic groups. This component is used to produce a single row per board/topic group. Further on in this subJob you will see that you use a tHashInput again (component 16) to use this dataset again. Here, you are identifying the board/topic information that you want; next time you use the tHashInput, you will be filtering by the board/topic records you aggregate here. To do this, configure the tAggregateRow as shown below:  Notice that you are aggregating by Type, Topic, and Board. The input/output schema can be seen below:  Notice that only Type, Topic, and Board are output. |
| 15 | tFlowToIterate | This component is used to iterate over the board/topic data returned by the tAggregateRow component. Each row is stored in a series of globalMap key/value pairs. The globalMap data will be used to filter records using component 17. Every component that follows the tFlowToIterate will fire before the next iteration of the tFlowToIterate is returned. This allows us to use the board/topic values to filter the data being returned by component 16 (tHashInput) that follows this component. An iterate link is used to join to the tHashInput that follows. |
| 17 | tFilterRow | This component is used to filter the data being returned by the tHashInput used for component 16. The filter data is stored in the globalMap. This data is set in the globalMap at component 15. What you need to do here is connect this component to the tHashInput preceding it and set the filter as shown below: Make sure Use advanced mode is selected (shown in the blue box). Then add the following code to the Advanced text box (shown in the red box): input_row.TYPE.equals(((String)globalMap.get("row2.TYPE"))) &&
(input_row.TOPIC!=null ? input_row.TOPIC.equals(((String)globalMap.get("row2.TOPIC"))) :
((String)globalMap.get("row2.TOPIC"))==null) &&
(input_row.BOARD!=null ? input_row.BOARD.equals(((String)globalMap.get("row2.BOARD"))) :
((String)globalMap.get("row2.BOARD"))==null)
You may need to check the globalMap key names, which have the following format: {row name}.{column name}
The row that feeds component 15 is the row name you need. Hopefully you are using the same column names. This is case sensitive, so be careful here. |
| 18 | tJavaFlex | This component is used to build the JSON message containing the data the user needs to be notified about, which is sent to the Slack API. There are three sections to this component:
Start Code String headerMessage = "";
if(((String)globalMap.get("row2.TOPIC"))!=null && ((String)globalMap.get("row2.BOARD"))!=null){
headerMessage = "Below are the unanswered posts for the board:
*"+org.apache.commons.lang3.StringEscapeUtils.escapeJson(((String)globalMap.get("row2.BOARD")))+"* and label:
*"+org.apache.commons.lang3.StringEscapeUtils.escapeJson(((String)globalMap.get("row2.TOPIC")))+"*";
}else if(((String)globalMap.get("row2.TOPIC"))!=null && ((String)globalMap.get("row2.BOARD"))==null){
headerMessage = "Below are the unanswered posts for the label:
*"+org.apache.commons.lang3.StringEscapeUtils.escapeJson(((String)globalMap.get("row2.TOPIC")))+"*";
}else{
headerMessage = "Below are the unanswered posts for the board:
*"+org.apache.commons.lang3.StringEscapeUtils.escapeJson(((String)globalMap.get("row2.BOARD")))+"*";
}
String block = "{\"channel\":\""+context.ChannelId+"\",\r\n\t\"blocks\":
[\r\n\t\t{\r\n\t\t\t\"type\": \"section\",\r\n\t\t\t\"text\":
{\r\n\t\t\t\t\"type\": \"mrkdwn\",\r\n\t\t\t\t\"text\":
\""+headerMessage+"\"\r\n\t\t\t}\r\n\t\t},\r\n\t\t{\r\n\t\t\t\"type\": \"divider\"\r\n\t\t}";
The Start Code builds the header section of the JSON message. It specifies the channel using the ChannelId context variable, and sets the header message that identifies the board, topic, or board and topic that the rest of the message applies to. Notice that you are using the org.apache.commons.lang3.StringEscapeUtils.escapeJson() method. This will ensure that your board and topic names are JSON compliant. The header section is stored in a newly created block String variable. This is used throughout the tJavaFlex component. Main Code block = block +",{\r\n\t\t\t\"type\": \"section\",\r\n\t\t\t\"text\":
{\r\n\t\t\t\t\"type\": \"mrkdwn\",\r\n\t\t\t\t\"text\":
\"<"+out1.URL+"|"+org.apache.commons.lang3.StringEscapeUtils.escapeJson
(out1.TITLE)+">\"\r\n\t\t\t}\r\n\t\t}";
This code adds a block to the JSON String. This represents a post that is related to the board/topic combination in the header. Remember to take a look at the Block Kit Builder if you want to change the JSON that is being built here. It’s always a good idea to generate the JSON in a tJava and output it to the System.out to make sure you get the code correct. The code returned in the System.out can be added back to the Block Kit Builder to check its validity. If it is valid, it will show you what will be seen in the display window. End Code block = block+",\r\n\t\t{\r\n\t\t\t\"type\": \"divider\"\r\n\t\t}\r\n\t]\r\n}";
globalMap.put("block", block);
This code is used to close the JSON String once the last block has been added, then store the block String in the globalMap to be used when you call the Slack API. |
| 19 | tFixedFlowInput | This component is used to supply our JSON payload to the tRestClient component following it. The tRestClient sends the JSON to the Slack API. The component is configured as shown below: Notice that a schema of two columns is configured: a null body column that is configured as a Document, and a string column that is populated with the following code: ((String)globalMap.get("block"))
This retrieves the JSON that was created in the previous tJavaFlex component. |
| 21 | tRestClient | The final component was described along with component 4, as they are configured in the exactly the same way. |
Running the Job
This Job is intended to be run by a parent Job that will call it for every user/channel that needs to be sent notifications, but it can also be run on its own if the required context variables are supplied. Since the data you are using as your source for the Slack messages will be different to the data in this example, how you run this is entirely up to you. In this case, I will run the child Job on its own to demonstrate the output.
To do this, I have set the following context variables in the Context tab of the Job:
- SlackOAuthAccessToken
- Username
- ChannelId
Once those are set, I simply select the Run tab and click Run. The output I get in my Slack app is shown below:
