Pass context variable or value from child job to parent job

[Anonymous]

Hi All,

I have a screnario where im doing file validation and moving the file from local to hdfs and creating external hive table in one job "job1" (trunjob1) , i am calculating the tot_rec_cnt in that subjob. Upon success or failure of subjob i want to trigger another subjob "job2" (trunjob) and want to pass the tot_rec_cnt from job1 to job2.

I have captured the tot_rec_cnt as a context variable and i want to pass the context to main job so that i can pass the same to my audit job(job2).

My job flow is like:

trunjob1(job1-filevalidation)  ----onsubjobok---- trunjob2(job2 audit job)

 |

onsubjoberror

|

trunjob2(job2 audit job)

I am calculating the tot_rec_cnt in trunjob1(job1) and want to pass the value to audit job. And adding the this, i want my audit job to called either for the success or failure of my file validation job

Kindly help to resolve this.

[Anonymous]

You should look at the tBufferOuput component which is usually used with tRunJob.

See https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/yh_GCK7WMATxbOSlI_Ck1Q

You send the values into the buffer in the child job.  You can link your tRunJob to a tMap or tLogRow or whatever components and read the value sent back by the child job into the tBufferOutput.

[Anonymous]

Hi ,
As of now I have done like that only.. I'm passing the value of job1 to buffer component and taking the value using tjavarow component from main job.
But the issue is whenever the job1 is success or gets failed, I want the tot_rec_cnt value to b passed to my audit job.
But as per this design,upon success of trunjob1, the value will be passed,correct?
How I can get the value in case of failure of job too?

[Anonymous]

You should design the child job being called by tRunJob1 in such a way that whether it succeeds or fails, there is always a value that is passed back.  The tPreJob and tPostJob always execute whether your main flow succeed or fails.  

You cannot handle the failure and count in a single subjob within the child job.  You need to count, maintain your count, and handle your errors in your subjob.  Then whatever the outcomes the childjob will return the count through the tBufferOutput.

[Anonymous]

Hi,
My child job(trunjob1) will read a file from ftp location and then do the file validation and post that I am calling another child job within that which will move the file in hdfs and create a hive external table.
I'm calculating tot_rec_cnt in file validation part.Please let me know how to pass that value upon failure of trunjob1?
Do I need to pass the tot_rec_cnt to some component and attached to tpostjob? Is that what you are telling?

[Anonymous]

Can you upload some screenshot please?

[Anonymous · Accepted solution]

There is another way of approaching this. You can set up a routine with static public variables. These will be available across your whole job (parent and child jobs). You can set this up really easily like below....

public class StaticVariables {
	
	public static String myString1;
	public static int myInt1;
}

....and access/assign these values using code like below (assuming your routine is called MyStaticVariables).....

routines.MyStaticVariables.myString1 = "Hello";
routines.MyStaticVariables.myInt1 = 100;

System.out.println(routines.MyStaticVariables.myString1);
System.out.println(routines.MyStaticVariables.myInt1+"");

[Anonymous]

Thanks..Let me try this approach..

[Anonymous]

Hi,

Sorry for the delay.. I have attached the main job screenshot where am  calling y file validation job(trunjob1)

and audit job(trunjob2,trunjob3).

I am calculating the tot_rec_cnt in trunjob1 which i need to pass to audit job.Upon success or failure,i want to pass the tot_rec_cnt to audit job.

As of now i have captured the tot_rec_cnt in tbufferoutput and passing the same in tjavarow. But i am afraid that if the file validation job gets failed ,in such situation this approach will fail.

Please advise

main_job.PNG

[Anonymous]

thanks rhall_2_0..it worked