Migrating a Talend Administration Center execution plan from on-premises to Talend in the cloud using Talend Cloud Migration Platform

TalendSolutionExpert · Aug 5, 2021 7:23:50 AM

A Talend Administration Center execution plan allows you to execute multiple Job Conductor tasks in a predefined workflow. The workflow is presented in a hierarchical view, where each task can execute a set of child tasks in parallel or sequentially.

The parallel approach executes all tasks simultaneously. Whereas in the sequential approach, the end status of the previous task determines when each task launches. You can add tasks in sequential order based on the following conditions

OnOk: Launches a child task if the parent task finalizes without error.
OnError: Launches a child task if the parent task finalizes with error.
After: Launches a child task after the parent task finalizes, regardless of its error status.

In this example, the execution plan consists of five tasks, including parallel and sequential tasks:

The root of the plan has two parallel tasks: “test_users” and “test_address”.
If the execution of parallel tasks is successful, “test_names” task executes.
If the execution fails, the “test_error” task launches.
Finally, the “test_emails” task launches after the “test_names” task, irrespective of the error status.

The plan also has a CRON-based trigger defined, trigger_plan, which executes every quarter on the 5th at 12:37.

How does Talend Cloud Migration Platform help with the migration of Talend Administration Center execution plans?

Talend Cloud Migration Platform collects and aggregates on-premises assets from Talend Administration Center and migrates them to Talend in the cloud. As a result, you can get a holistic view of all of the on-premises data from different Talend Administration Center environments in a centralized location.

To do so, you must register Talend Administration Center servers in Talend Cloud Migration Platform by providing their Talend Administration Center web URL endpoint and database connection details. Once the servers are registered, Talend Cloud Migration Platform collects all the assets, including execution plans and their respective tasks. The aggregated list, across all of the registered Talend Administration Center servers, is then displayed in Talend Cloud Migration Platform.

You can view the execution plans and tasks in their respective Talend Cloud Migration Platform UI pages.

Migrate Talend Administration Center tasks and execution plans from on-premises to Talend in the cloud by performing the following five steps:

Select the source Talend Administration Center environment, in this case, TacDev, and then click the Cloud button.
Select the destination Talend Management Console Environment and its respective Workspace from the drop-down menu. The destination environment is where you will migrate and execute the assets.
1. Specify a target runtime engine. You can deploy a Talend Management Console plan on three types of runtime engines:
  The lists of Remote Engines and Remote Engine Clusters are refreshed based on the selected Talend Management Console environment and workspace.
  - Cloud Engine
  - Remote Engine
  - Cluster of Remote Engines
2. Click the TASKS button to proceed to the next step.
Select the execution plan and the corresponding tasks.

The left side of the UI represents Talend on-premises plan and tasks tree from Talend Administration Center, and the right side represents the plan and tasks tree in Talend Management Console.

The execution plans and tasks are sorted alphabetically in the Talend Administration Center tree. The tasks in the Talend Administration Center tree are grouped by project name. Each task contains an artifact name with a version and any defined triggers.

In the Talend Administration Center tree, only tasks of the type ARTIFACT_TASK are listed. These task types are the only ones suitable for migration as they are assigned with Artifact GAV information. Artifact GAV (GroupId, ArtifactId, Version) is necessary for Talend Cloud Migration Platform to locate the task artifact from Artifact Repository for downloading from Talend Administration Center and later to upload to Talend Management Console.

You can select the plan by clicking its name in the Talend on-premises tree (the Talend Administration Center execution plans and tasks). With this selection, the corresponding tasks and triggers get automatically selected for the migration.

Talend Management Console supports only one trigger or schedule per task, so if a task defines multiple triggers, not all of them will migrate. Instead, multi-trigger tasks migrate with the trigger set as manual, and a warning message "More than one trigger defined" displays after the migration.
1. Click the arrow facing right between the two trees to move the execution plans and tasks.
2. Click the Review button to continue with the migration
Review the assets you are migrating, Talend Management Console environment, Workspace, and Runtime details.

By default, Talend Cloud Migration Platform migrates the artifact and trigger definitions along with execution plan definitions and the tasks.

If the Artifact toggle is On and an artifact with the same name already exists in Talend Management Console, a new version of the artifact is created. You can disable this type of occurrence (that is, disable migration of artifacts with the same name) by toggling the option Off. A similar option is also available for the migration of trigger definitions.

You can also add comments about the migration.

When you finish reviewing the information, click the MIGRATE button to start the migration process.

If multiple tasks share the same artifacts in the execution plan, you cannot proceed with the migration, and you'll see the following message.

Use one of the following workarounds to bypass this limitation:
- Toggle the Artifact option to Off. However, this setting applies to all the tasks and will not migrate artifacts with the same name even if multiple tasks did not share them.
- Migrate the affected tasks before the plan, and then migrate the plan with the Artifact option toggled Off.
Notice that the migration occurs simultaneously for all the selected Plans and Tasks. The migration process might take some time if the size of the artifact is large.

A green check box displays to the right of each successful asset migration. The left panel of the UI displays the completed percentage of the migration process and summarizes the results.

The migrated artifact of the task is replaced with the artifact name and version in the Talend Management Console tree.

The execution plan in Talend Management Console is created as steps, and the Talend Administration Center plan trigger is mapped to a Talend Management Console trigger.

Click the CLOSE button to redirect to the Timeline UI, which displays the history of all migrations in chronological order.

Talend Cloud Portal

You can view the newly migrated execution plan and tasks in the Talend Management Console UI, and easily confirm that they migrated successfully.

The tasks in the plan are arranged sequentially and represented in the form of steps:

Step 1 consists of parallel tasks (test_users, test_address) and failure handler (test_error).
Step 2 launches the test_name task only if parallel tasks in Step 1 are executed.
Step 3 executes test_emails after Step 2, irrespective of the error status.

Talend Administration Center plan CRON trigger, which executes Every quarter on 5th at 12:37 is mapped to TMC MONTHLY trigger which executes Every 3 month, on 5th at 12:37.