I have Talend Cloud with Remote Engine Gen2 on Azure. Previously the team was using Talend Data quality and Talend DI on-premise using remote engine (old version, pre-gen2). Are there any issues with running legacy jobs on a gen2 remote engine? I have seen conflicting information that says some legacy jobs will not run on gen2 and some people have said everything that works on the old engine will work in the gen2.
Has anyone run into a situation where you need both or can all the engines be updated to Gen2 with full backwards compatability.?
The Remote Engines (RE) and the Remote Engines Gen 2 (REG2) server different purposes. The RE did come out before the Gen2 but they do not currently process the same types of jobs. In summary, the RE is for data integration jobs created in Talend Studio and REG2 is used for Talend Pipeline Designer, Talend Data Preparation, and Talend Data Inventory.
For additional details please see the descriptions below:
Remote Engines (RE)
A capability in Talend Cloud that allows companies to securely run data integration jobs natively from cloud to cloud,
on-premises to cloud or cloud to on-premises completely within customer's environment for enhanced performance
and security, without transferring the data through Talend cloud's cloud engines.
-A Java-based runtime (similar to a Cloud Engine) to execute Talend Jobs on-premises or on another cloud platform that is controlled by the customer
-Remote Engines allow you to run Job, Route, and data service tasks.
-Data service and Route Microservices tasks can only be deployed on Remote Engines. OSGi type deployments require that Talend Runtime 7.1.1 version or higher is installed and running on the same machine as the Talend Remote Engine.
-By default, maximum 3 different tasks can run in parallel on the same engine. However this is a modifiable configuration
-Remote Engines can be grouped together to form clusters called “Remote Engine Cluster”. Remote Engines added to a cluster cannot be used to execute tasks directly from Talend Studio.
A Remote Engine Gen2 is a secure execution engine on which you can safely execute Data Pipelines (i.e. data flows designed using Talend Cloud Pipeline Designer). It allows you to have control over your execution environment and resources as you are able to create and configure the engine in your own environment (Virtual Private Cloud or on
premises).
This engine was previously referred to as ‘Remote Engines for Pipelines’ and was renamed to ‘Remote Engine Gen 2’
during H1/2020. It is a docker-based runtime to execute Data Pipelines on-premises or on another cloud platform that is controlled by the customer.
A Remote Engine Gen2 ensures:
-Data processing in a safe and secure environment as Talend never has access to the data and resources of
your pipelines
-Optimal performance and security by increasing the data locality instead of moving large data to computation
