topic Re: Rollback/Commit a job with serial and parallel connections in Talend Studio

Rollback/Commit a job with serial and parallel connections

Anonymous — Sat, 16 Nov 2024 09:33:45 GMT

Hi,

I have a job that has many other jobs connected. The basic design looks like this

tprejob --> tmysqlconnection

job1 --onsubjobok--> job2 --onsubjobok-- job3

tpostjob -- job5

job5 has many other jobs inside it and all those jobs in side of job5 run in parallel

Now, my question is how do I rollback all data if a job fails either in serial connection or parallel connection? Similarly, the commit too?

The work is on mysql database

Thanks
Rathi

Re: Rollback/Commit a job with serial and parallel connections

Anonymous — Wed, 05 Jul 2017 10:08:56 GMT

I will advise moving job5 into your main job after the job3 with an onSubjobOk. Hence,all your parallel logic should be part of the main job logic.

Prejob and PostJob should be short and sweet. You just open your connection in PreJob and Close your connection in PostJob.

Commit and Rollback functionality should be part of the main job.

If all your DB components are using the same tMySQLConnection, you should be able to put a rollback on it. You should use a shared db connection (specify the name everywhere in all connection components) and all your childjob should be running within the same process.

Re: Rollback/Commit a job with serial and parallel connections

Anonymous — Wed, 05 Jul 2017 11:39:47 GMT

Hi iburtally,

Thanks for the response. I get the point to move job3 to main job

In the series of jobs where do I put the commit and rollback, this is what I am unable to get? I have lets say 10 jobs connected to each other, if a single job fails I want all the jobs to do a rollback, so where in should I put the rollback and commit?

Thanks

Rathi

Re: Rollback/Commit a job with serial and parallel connections

Anonymous — Wed, 05 Jul 2017 15:07:41 GMT

You should put it at the complete end of the main job, and maybe track a status variable to say whether an error happened or not. Depending on the status variable you will call commit or rollback on all the logic.

Re: Rollback/Commit a job with serial and parallel connections

Anonymous — Thu, 06 Jul 2017 11:17:33 GMT

Thanks.

I am trying this and have a few questions

I create a main job and set the tmysqlconnection and give it the name mysql_shared_db. Now in the child job I pull another tmysqlconnection and just give the details of the main connection in the main job i.e., mysql_shared_db. Now, when I pull a tmysqlinput in the child job and use the mysql_shared_db details, the connection is not recognized.

The below picture is the main job and below that is the child job. May I please know what is my mistake here?

main jobchild job

Thanks

Rathi