Talend ESB: Managing Talend microservices with Istio service mesh on Kubernetes

TalendSolutionExpert — Tue, 23 Jan 2024 02:35:30 GMT

Content:

The Talend Community Knowledge Base article, Containerization and orchestration of Talend microservices with Docker and Kubernetes shows you how to:

Build a demo Customer service as a Talend microservice container image
Generate Kubernetes resources for the Customer service using the f8-m-p plugin
Deploy the Customer service on a Kubernetes cluster using the f8-m-p
Invoke the Customer service using a ClusterIP

However, in Production environments, it is more likely you'll need to:

Expose a few microservices to external endpoints using an external load balancer
Route traffic dynamically to several internal endpoints running in a distributed cluster
Handle faults, timeouts, and perform retries
Roll incremental changes or bug fixes without service outage
Deploy multiple versions and route incoming traffic based on a match criteria
Observe logs, metrics, and health of the services

This article provides another approach to deploying Talend microservices to a distributed Kubernetes cluster using the kubectl utility, and leveraging Istio to manage some of the production-grade requirements for microservices.

Prerequisites

Talend 7.2 or above
Internet access
Access to a Docker repository
Install a MySQL database
Set up Kubernetes cluster and install Istio
Download the sources for the project available in the Microservices-JobDesigns.zip file
Download the demo Kubernetes resource ZIP file (attached to the article) for one of the following cloud platforms:
- Demo Kubernetes resource files for AKS
- Demo Kubernetes resource files for GKE
- Demo Kubernetes resource files for EKS

Environment

Before you can understand some of the Istio features, you need to set up a Kubernetes cluster and install Istio, by following the platform-specific instructions using one of the following Talend Community Knowledge Base (KB) articles:

Follow the Launching a standard Azure Kubernetes Service (AKS) and Installing Istio with Helm on AKS sections in the Managing Talend microservices with Istio service mesh on Azure Kubernetes Service (AKS) article.
Follow the Launch a standard Google Kubernetes Engine (GKE) and Installing Istio with Helm on GKE sections in the Managing Talend microservices with Istio service mesh on Google Kubernetes Engine (GKE) article.
Follow the Launching an Elastic Kubernetes Service (EKS) and Installing Istio with Helm on EKS sections in the Managing Talend microservices with Istio service mesh on Amazon Elastic Kubernetes Service (EKS) article.

Use case: Order processing

Typically, a request passes through different stages during the order processing lifecycle, such as Create, Get, and Amend.

The following diagram showcases a few of those stages, the sequence of steps, and the services involved in a CreateOrder and GetOrder workflow.

In this scenario, you're building a company's retail car purchasing microservice where:

The CreateOrder workflow is responsible for processing all the new orders placed by customers using different sales channels.
The GetOrder workflow retrieves the details and status of a previously placed order.

Creating a mock order

This section shows you how to create a mock order (referred to as a mock order because it's not a real order) with a sample MySQL database that contains information about classic car models purchased by customers.

Launch an RDS MySQL instance.
Import the mysql-sampledatabase-classicmodels.sql file (attached to this article) to a folder.

sql connect INSTANCE_NAME --user=root --quiet

Create the classicmodels MySQL database by using the source command:
```
source file_location/mysql-sampledatabase-classicmodels.sql
```
After the classicmodels database is created, observe the relationship between the customers, orders, and products entities.

Implementing the GetOrder

The GetOrder workflow retrieves all the orders made by a customer and displays the status of all the orders. This section shows you how to implement the GetOrder services as a Talend microservices.

Microservice Orders

Launch Talend Studio, then import the Jobs in the Microservices-JobDesigns.zip (attached to this article) file.
Open the Orders microservice.
Click the Job tab, then click Deployment and notice that the Build Type is set to Microservice for ESB.
Observe the Job design:
- Orders service is a REST data service
- Fetches all the order details from the database using customerNumber
- Uses three different context environments namely STUDIO, QA, and PROD
Run the Job in Studio. By default, the service listens on port 8090.

Microservice Customers

Open the Customers microservice.
Click the Job tab, then click Deployment and notice that the Build Type is set to Microservice for ESB.
Observe the Job design:
- Customers service is a REST data service
- Fetches all the customer details by customerNumber
- Invokes the Order service and all the orders created by the customer
- Uses three different context environments namely STUDIO, QA, and PROD
- In the context variables, observe the endpoint of the REST client, for example, the ORDER_SERVICE_CLIENT_URL parameter
- The host and port of the ORDER_SERVICE_CLIENT_URL changes for every environment
- In the PROD context the host is set with a logical host name, for example, order-service
Run the Job in Studio. By default, the service listens on port 8090 and fails if the port is already in use.

Build the GetOrder services

In the Studio, right-click the Customers microservice, select Build Job, and save the build to a folder demo.
Follow the process in Step 1, and build a Job for Orders.
Launch the Windows Command Prompt and move them into the demo folder.
Follow the steps in the next section and run both the microservices.

Running the GetOrder service as standalone microservice with QA context

Run the Orders service:

Java -jar Orders_0.1.jar --server.port=8065 --spring.config.location=classpath:config/contexts/ --spring.config.name=QA

Run the Customers service:

Java -jar Customers_0.1.jar --server.port=8066 --spring.config.location=classpath:config/contexts/ --spring.config.name=QA

Invoke the Customers service using Talend API Tester or in a browser, enter the following URL and click the Send button:
```
http://localhost:8066/services/customer/103
```
Notice that the GetOrder service returns all the orders created by a customer upon call.

Use case: Order processing with Istio and Kubernetes

Istio is an open-source framework that provides a service mesh for microservices and can be installed on Kubernetes.

Why Istio with GetOrder Services?

Istio can provide many additional capabilities to the Talend microservices by following a few best practices and without any additional changes to the Job designed in Talend Studio.

Adding Istio capabilities for GetOrder services

With Istio sidecar injection, a sidecar proxy Envoy is deployed in front of the GetOrder services (customer-service, order-service) and provides Istio Envoy capabilities. For more information on Envoy and its features, read the Istio Envoy documentation.

Service Discovery

Service discovery explains how a service register and deregister its endpoint when deployed on a distributed cluster and how the clients/services can dynamically locate the endpoint URLs. Istio Pilot has a built-in Kubernetes adapter and watches the Kubernetes API server for pod registrations and deregistrations.

Note: Istio service discovery leverages the discovery features provided by platforms like Kubernetes.

The diagram above shows how Istio:

Adds Envoy proxy in front of the Talend microservice application pods
Service discovery works for locating the services internally with in the mesh

For example, the customer-service Pods in namespace talend-istio (1) dynamically discovers the endpoint URL of the order-service (4) by making a name lookup (order -service or order-service.talend-istio) using the Istio Kubernetes adapter (2) and the Envoy proxy (3).

For more information, read Discovering services in the Kubernetes using DNS documentation.

Traffic management

The advantage of Istio Traffic management:

It allows developers and deployment teams to easily configure routing rules and control the flow of traffic and API calls between services.
Operation teams can apply new rules or modifications without stopping the services.
Istio Pilot watches for changes in routing rules and dynamically propagates new changes to the Envoy proxy.

Gateways

The Gateway manages external Ingress traffic to our service mesh. It supports multiple protocols and can be configured to allow traffic on specific ports.

Virtual services

A virtual service allows to configure an ordered list of routing rules and decides control how the Envoy proxy should route the traffic.

The following diagram explains how:

External traffic is allowed to the service mesh using the Ingress Gateway
Define routing rules in a virtual service
Apply rules and impose match conditions on the request URI
Route traffic to one or more destinations

Refer to this VirtualService configuration settings in the Service_Discovery.yaml file.

Destination rules

After enforcing the routing rules on the incoming traffic with virtual services, additional policies can be applied using destination rules.

For example, the incoming traffic to order-service can be routed to different versions (v1, v2) of the same service using subsets by specifying match conditions and weights.

Refer to this VirtualService and DestinationRule configuration entries in the CanaryDeployments.yaml file.

Service Entry

ServiceEntry enables adding additional entries into the Istio internal service registry so that auto-discovered services in the mesh can access/route to these manually specified services.

In the GetOrder use case, the Talend microservice residing inside the service mesh need access to an external RDS MySQL instance.

Refer to this ServiceEntry configuration in the Service_Discovery.yaml file.

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mysql-external
spec:
hosts:
- istio-customer-db.some_id.eu-central-1.rds.amazonaws.com
ports:
- name: tcp
number: 3306
protocol: tcp
location: MESH_EXTERNAL
resolution: DNS

Logical architecture with Istio and Talend microservices

The microservices in the GetOrder use case are deployed to Kubernetes using Istio sidecar injection.

The following diagram depicts a high-level logical architecture of the complete setup.

Publishing GetOrder microservices to a container registry

To publish the GetOrder microservices to a container registry follow the instructions in the applicable article:

Deploying GetOrder microservices with Istio on Kubernetes

Deploying the GetOrder services on a Kubernetes cluster requires the following resources:

Kubernetes resource files (YAML)
Cloud / SDK shell
Kubernetes command-line tool (kubectl)

These are available in the demo Kubernetes resource file you download in the Prerequisites section of this article.

Set Talend context environment

The GetOrder services used in this article use Talend context environment and context variables, for example, the Prod context environment and context variables are used in this article for microservices deployments on a Kubernetes cluster.

Unzip the demo Kubernetes resource file you downloaded in the Prerequisites section of this article.

Open the resources files for one of the Kubernetes clusters.
Search for the API type Deployment.
Observe the value of the spring.config.location property.

spring.config.location is set with the classpath location of Talend context environment properties file, for example, PROD.properties
```
command: ["/bin/sh", "/maven/Customers/Customers_run.sh"]
args: ["--spring.config.location=classpath:config/contexts/PROD.properties"]
```

Enable Istio sidecar injection at the namespace level

Connect to the cloud shell, create a namespace called talend-istio:
```
kubectl create ns talend-istio
```

Enable Istio sidecars injection on the new namespace:

kubectl label namespace talend-istio istio-injection=enabled

Deploy the GetOrder services using a Kubernetes resource file

Using the kubectl utility, execute the apply command along with the resource file.
```
kubectl apply -n talend-istio -f FILE_NAME
```
From the cloud / SDK shell, observe that the Status of the pods in the talend-istio namespace is Running.
```
kubectl get pods -n talend-istio
```
Observe the External-IP associated with the istio-ingressgateway.
```
kubectl -n istio-system get service istio-ingressgateway
```
Note: The IP address assigned to istio-ingressgateway is the same as the IP address of the external LoadBalancer.

Undeploy the GetOrder services using a Kubernetes resource file

Using the kubectl utility, execute the delete command along with the resource file.
```
kubectl delete -n talend-istio -f FILE_NAME
```
From the cloud / SDK shell, observe that the pods in the talend-istio namespace were deleted.
```
kubectl get pods -n talend-istio
```

Demo

The demo uses Istio Dashboards for Monitoring, execute the platform-specific instructions, and open the dashboards for Kiali, Grafana, and Jaeger:

Launching Istio monitoring dashboards for GKE
Launching Istio monitoring dashboards for AKS
Launching Istio monitoring dashboards for AWS

Service discovery and routing

Using the kubectl utility, execute the apply command along with the resource file.
```
kubectl apply -n talend-istio -f Service_Discovery.yaml
```
Before executing the apply command download and modify the cloud-specific template files, and make the following changes:
1. Search for the API type Deployment, replace the value in the image property with your registry location URL:
```
image: REPLACE_WITH_YOUR_RESPOSITORY_URL/k8s_microservice/customer/microservices/orders:0.2.0
```
  If you publish the images to Azure Container Registry, then Use Kubernetes Secrets in the resource files in the API of type Deployment, for more detailed steps follow the section Authentication with Azure Container Registry from AKS using Service principal.
2. Search for the API type ServiceEntry, replace the hosts and number property values with the URL and port number of an external database.
```
hosts:
- istio-customer-db.some_id.eu-central-1.rds.amazonaws.com
ports:
- name: tcp
number: 3306
```
From the cloud / SDK shell, observe that the Status of the pods in the talend-istio namespace is Running.
```
kubectl get pods -n talend-istio
```
From your local machine (Windows / Linux), connect to Kiali, observe that there are two applications in the talend-istio namespace, and there is no traffic.
Invoke the Customers service endpoint using Talend API Tester, enter the following URL, click the Send button, and observe the response.
```
http://REPLACE_WITH_YOUR_INGRESS_EXTERNAL_IP/customer/103
```
Create more load on the Customers service, invoke the Customers service endpoint by clicking the Send button 4 to 5 times. Notice that the dashboard detects some traffic.
In the Dashboard, from the menu on the left, click Graph, then select the Workload graph.
1. In the Workload graph, observe the last 5 minutes of traffic.
2. Notice that the services are arranged in a sequence following their order of invocation, for example, istio-ingressgateway > customer-service proxy > customer-service application > istio-customer-db > order-service proxy > then order-service application.
3. Observe that the HTTP traffic is green, indicating there are no failures.
Click the Namespace down arrow next talend-istio, and add the istio-system namespace.
- Review the Istio service graph, and notice that the istio-telemetry service is added to the graph
- istio-telemetry gathers metrics data from the sidecars customer-service and order-service
From your local machine (Windows / Linux), open the Google Cloud SDK shell, then using local port forwarding, connect to Grafana. In the Istio Workload Dashboard, observe the request/response duration and size.
From your local machine (Windows / Linux), connect to Jaeger. Find Traces for customer-service, observe the dependency between the services and latency.

Note: Applications must manually propagate appropriate b3 HTTP headers for correlating spans into a single trace. For more information, read Trace context propagation and Zipkin b3-propagation

Recovery failures

This section helps you understand how Istio handles recovery failures such as faults, timeouts, and retries by enhancing the configuration from the Service discovery and routing section of this article.

In this scenario, assume the order-service runs CPU intensive tasks and not accepting new requests. To understand this scenario, configure the Virtual service as below:

Configure Istio to create a delay of two secs before routing the request to order-service
Customer-service timeouts due to the latency induced by order-service
On timeout, customer-service performs a retry

Using the kubectl utility, execute the delete command and clean the previous deployments:
```
kubectl delete -n talend-istio -f Service_Discovery.yaml
```
Using the kubectl utility, execute the apply command along with the resource file:
```
kubectl apply -n talend-istio -f RecoveryFailures.yaml
```
Note: Before executing the apply command, modify the YAML file for environment-specific changes as follows:
1. Search for the API type Deployment, replace the value in the image property with your registry location URL.
```
image: eu.gcr.io/customer-success-account_id/k8s_microservice/customer/microservices/orders:0.2.0
```
2. Search for the API type ServiceEntry, replace the hosts, and number property values with the URL and port number of an external database.
```
hosts:
- istio-customer-db.some_id.eu-central-1.rds.amazonaws.com
ports:
- name: tcp
number: 3306
```
From the cloud / SDK shell, observe that the Status of the pods in the talend-istio namespace is Running.
```
kubectl get pods -n talend-istio
```
Invoke the Customer service endpoint using Talend API Tester, enter the following URL, and observe the response.
```
http://REPLACE_WITH_YOUR_INGRESS_EXTERNAL_IP/customer/103
```
Create more load on the Customer service by clicking the customer-service endpoints 4 to 5 times. Notice that the dashboard detects some traffic.
From your local machine (Windows / Linux), open the Google Cloud SDK shell, then using local port forwarding, connect to Kiali.
Open the Versioned app graph, observe the customer-service is Red and 10% of the HTTP traffic fails due to timeouts induced on order-service.

Canary deployments

Using the kubectl utility, execute the delete command and clean the previous deployments.
```
kubectl delete -n talend-istio -f RecoveryFailures.yaml
```
Using the kubectl utility, execute the apply command along with the resource file.
```
kubectl apply -n talend-istio -f CanaryDeployments.yaml
```
Note: Before executing the apply command, modify the YAML file for environment-specific changes as follows:
1. Search for the API type Deployment, replace the value in the image property with your registry location URL.
```
image: eu.gcr.io/customer-success-account_id/k8s_microservice/customer/microservices/orders:0.2.0
```
2. Search for the API type ServiceEntry, replace the hosts, and number property values with the URL and port number of an external database.
```
hosts:
- istio-customer-db.some_id.eu-central-1.rds.amazonaws.com
ports:
- name: tcp
number: 3306
```
From the cloud / SDK shell, observe that the Status of the pods in the talend-istio namespace is Running.
```
kubectl get pods -n talend-istio
```
Invoke the Customer service endpoint using Talend API Tester, enter the following URL, and observe the response.
```
http://REPLACE_WITH_YOUR_INGRESS_EXTERNAL_IP/customer/103
```
Create more load on the Customer service by clicking the customer-service endpoints 4 to 5 times. Notice that the dashboard detects some traffic.
From your local machine (Windows / Linux), open the Google Cloud SDK shell, then using local port forwarding connect to Kiali.
From your local machine (Windows / Linux), open the Google Cloud SDK shell, then using local port forwarding, connect to Grafana.
In the Istio Workload Dashboard, observe the traffic was split into two versions of the order-service.

Conclusion

This article explains how Talend integrates with distributed platforms like Kubernetes and how you can leverage Istio for managing traffic, service discovery, monitoring, and tracing services in a service mesh. Services are designed, developed, and published with the help of Talend Studio.

article Talend ESB: Managing Talend microservices with Istio service mesh on Kubernetes in Official Support Articles