Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Jan 22, 2024 9:35:30 PM
Jan 21, 2021 9:20:31 AM
This article provides a quick checklist of items required to prepare your first environment to use Talend Cloud.
For more information, see the Talend Cloud Getting Started Guide available in the Talend Help Center.
Content:
Hardware preparation
Talend Cloud solution requires instances (servers).
A minimum of one instance is needed for Talend Studio as it is the main product used to design Jobs to handle your data.
When the execution of a Job needs to be done on your network (for example, security reasons), install and configure an execution server with the Talend Remote Engine per environment.
For both instances, Talend recommends using the following configuration:
|
Workstation/Server Role |
Machines |
Type |
OS |
CPU |
RAM |
SSD Disk Size |
|
Client PC for Talend Studio Installation |
1 for each user |
Desktop |
Windows/ Linux/ Mac |
Intel i7 Processor 4 Cores or equivalent |
16 GB |
500 GB |
|
Remote Engines Servers |
1 for each environment |
IaaS |
Windows/Linux |
8 Cores minimum |
32 GB RAM minimum, 128 GB recommended |
500+ GB |
Recommended sizing for a hybrid installation for the Data Preparation and Data Stewardship tools:
|
Workstation/Server Role |
Machines |
Type |
OS |
CPU |
RAM |
SSD Disk Size |
|
Data Preparation and Data Stewardship |
1 or 2 (for HA) |
IaaS |
Windows/ Linux |
4 Cores minimum |
16 GB |
500+ GB |
Architecture
Instance prerequisites - Java
All Talend products are based on Java to run, which needs to be installed and configured before installing and configuring the products. Based on the security enhancements, Talend recommends installing any version of Java 8 with the sub-version higher than 131.
It also requires having the environment variables configured, such as the JAVA_HOME variable.
Note: If Talend Studio is installed with the Installer downloaded from Talend Cloud, a specific version of Java (Zulu) is shipped, and no other configuration is needed for Talend Studio.
Ports
Many interactions and communications are done between Talend products and the different systems that interact with them. Connectivity interaction is the main challenge when working in the Talend Cloud environment.
These interactions can be divided into different parts:
- Management:
- The different Talend products need to exchange information to have the environment working, such as:
- Retrieving the license for Talend Studio from Talend Management Console
- Publishing Jobs from Talend Studio to Talend Management Console
- Orchestration and executing of Jobs from Talend Management Console to the Talend Remote Engine
- Executing a Job from Talend Studio to the Talend Remote Engine
- The different Talend products need to exchange information to have the environment working, such as:
- Execution:
- Talend products are also communicating with other systems to manage and handle data, such as:
- Ingesting data from a file system
- Ingestion data from a database
- Pushing back data on another data source
- Talend products are also communicating with other systems to manage and handle data, such as:
All of these systems need to communicate with each other.
Table summary of all the ports:
|
Components |
URL Destination |
Port |
Inbound/ Outbound |
Usage |
|
Studio |
update.talend.com |
443 |
Outbound |
For downloading additional packages such as Bonita BPM Integration, Talend Metadata Bridge, and upgrades from Talend Studio tools |
|
Studio |
talend-update.talend.com |
443 |
Outbound |
For downloading libraries in Talend Studio(mainly for components) |
|
Studio |
GIT URL |
<To be filled> |
Outbound |
Studio connecting to Git to download and upload ETL Job files |
|
Studio |
Talend Cloud SaaS URL: |
443 |
Outbound |
Studio connecting to Talend Cloud for authentication and license, requesting configuration information, and Job publishing |
|
Studio |
All source and destination data sources and internet |
<To be filled> |
Outbound |
Studio connects to all data sources and destinations for building ETL pipelines |
|
Studio |
Data Preparation |
9999 |
Outbound |
Access from Studio to Data Preparation server for running the cleaning recipe |
|
Studio |
Data Stewardship |
19999 |
Outbound |
Access from Studio to Data Stewardship server for pushing exception records as part of campaigns |
|
Remote Engine (RE) |
Studio instance |
8003 |
Inbound |
Studio connections to RE on these ports: |
|
Remote Engine (RE) |
Talend Cloud SaaS |
443 |
Outbound |
RE outward call to the Talend Cloud for pulling the Job artifacts to run as per schedule Details of the URL required: https://help.talend.com/reader/vfK0vMfgfqfNpuDhC5QOiA/r1nykvHZuwPkPQn1CujqVQ https://accounts-iam.<region>.cloud.talend.com |
|
Remote Engine (RE) |
All source and destination data sources and internet |
<To be filled> |
Outbound |
RE connectivity to all data sources and destinations |
|
Data Stewardship |
localhost |
19999 |
Inbound |
Web URL accessed from browsers |
|
Data Preparation |
localhost |
9999 |
Inbound |
Web URL accessed from browsers |
|
Data Preparation |
All Data Sources |
<To be filled> |
Outbound |
Connectivity to all data sets which need to be cleaned and profiled |
Proxy and certificates
Talend products (Studio and Remote Engine) need to establish a connection with Talend Management Console to communicate and retrieve required information such as the license file, authentication of a user, execution of a task, and more.
This connection might be interrupted by a proxy installed by your company. Enable this communication (in and out) by adding the proxy's configuration information to the products.
The required information is:
- Hostname
- Port
- Username
- Password
All the services provided by Talend Management Console are secured with TLS (HTTPS calls). This security is providing certificates to the services using it. In most cases, these certificates are self-resolved by the services or tools using them.
If your security team is replacing the original certificates to handle the communications and adding another layer of security to access the services on the internet (for example, with ZScaler), some configuration is required. Those specific certificates need to be imported to the Java certificate manager (for example, certificate import using keytools).
In other cases, no configuration is needed to access these services by the Talend products (if the instance whitelists the URL).
Git integration
Talend Studio is a Java code generator. All the code created can be managed by a version control system such as Git to store all your project data (Jobs, database connections, Routines) in a shared repository.
Some information and permissions are needed to use it with the Talend products.
In Talend Management Console, where you define your project, you'll need the URL of an existing repository.
In Talend Studio, you'll need the connection credentials to access the repository and the project.
The required information is:
- URL of the Git repository
- Username of the user (or service account)
- Password of the user (or token)
Note:
-
The connection to the Git repository is made only at the Studio level. At the Talend Management Console level, it is a reference or metadata sent to Talend Studio.
-
For Git: If the SSO is activated on the account, the token needs to be generated within Git and configured (enablement of SSO) to use it with Talend Studio.
Software preparation
All the different Talend products can be found on the Talend Cloud platform.
|
Product name |
Instance destination |
Download instruction |
|
Talend Studio |
Client laptop for Talend Studio instance |
Talend software can be downloaded from Talend Cloud SaaS (Downloads page) - with OS version choice |
|
Talend Remote Engine |
Remote Engine instance |
Talend software can be downloaded from Talend Cloud SaaS (Downloads page) - with OS version choice |
|
Java environments for all Studio and instances |
All instances |
Official Oracle Java website |
