topic Re: tRunJob does not transfer Context variables when "Use dynamic job" sel in Talend Studio

tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Fri, 02 Mar 2012 03:21:59 GMT

tRunJob does not transfer Context variables when "use dynamic job" selected.
I am running into issue of transfering context variables to SubJob, when tRunJob is set as "Use dynamic job".
The goal is to set up context variables in job "ContextLoad" and pass them through to dynamically changing Subjob.
Scenario:
1) create Main job called "ContextLoad" and add 2 context variables: DataFileFolder, DBName
2) drop 2 components into this job: tJava and tRunJob. connect tJava with tRunJob using "OnSubjobOK"
3) Enter code below to tJava component:

// Define Context variables
//
context.setProperty("DataFileFolder","D:/tmp/Test/");
context.setProperty("DBName","testingDB");
System.out.println("BEFORE the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));

4) Create another Job called "Display_context_vars" and add 2 context variables: DataFileFolder, DBName
5) Drop one tJava componenet into this job.
6) Enter code below into this component:

System.out.println("INSIDE of the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));
7) Save job "Display_context_vars"
😎 go back to job "ContextLoad"
9) configure tJobRun componenet as below:
- check on "Use dynamic job"
- enter "Display_context_vars" to "Context Job" text field
- click "..." button beside "Job" and select job "Display_context_vars" from the list
- click "Transmit whole context" check box
10) run job "ContextLoad". If you received same results as me, than you were able to reproduce my issue:
Starting job ContextLoad at 18:13 01/03/2012.
connecting to socket on port 3991
connected
BEFORE the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testingDB
INSIDE of the SUBJOB:
Data File Folder: null
DBName: null
disconnected
Job ContextLoad ended at 18:13 01/03/2012.

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Fri, 02 Mar 2012 03:54:32 GMT

Hi
This is due to the code in tJava of "ContextLoad".
Set it as follow.

context.DataFileFolder="D:/tmp/Test/";
context.DBName="testingDB";

System.out.println("BEFORE the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));

Result is
connecting to socket on port 3908
connected
BEFORE the SUBJOB:
Data File Folder: null
DBName: null
INSIDE of the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testingDB
disconnected
Set it as follow.

context.DataFileFolder="D:/tmp/Test/";
context.DBName="testingDB";
context.setProperty("DataFileFolder","D:/tmp/Test/");
context.setProperty("DBName","testingDB");
System.out.println("BEFORE the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));

Result is
connecting to socket on port 3511
connected
BEFORE the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testingDB
INSIDE of the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testingDB
disconnected
Regards,
Pedro

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Fri, 02 Mar 2012 20:51:51 GMT

Thank you Pedro for your quick response!
This is a very interesting concept - thank you for the clarification.
The job works as you explained it would.
Now, my next step is to load my context from database and transfer it over to Sub job.
All context variables are setup in Repository as Context Group. This Context Group is used in both "ContextLoad" and "DisplayContext" jobs.
This is how I have modified my job "ContextLoad":
1) add 3 new components:
- tConnection ( for database or it could be file - what ever works for you)
- tInput (specific to your choice above) I need 4 context variables to be loaded by this input: DBName, DataFileFolder, JobName and JobRunID
- tContextLoad component (click "Print Operation" to view loaded context)
2) Modify tJava component code as below:
System.out.println("BEFORE the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));
System.out.println("JobName: " + context.getProperty("JobName"));
System.out.println("JobRunID: " + context.getProperty("JobRunID"));
context.JobRunID="2";
System.out.println("BEFORE the SUBJOB and after changing JobRunID context:");
System.out.println("JobRunID: " + context.getProperty("JobRunID"));

Now Modify subjob "DisplayContext":
3) modify code in tJava component:
System.out.println("INSIDE of the SUBJOB:");
System.out.println("Data File Folder: " + context.getProperty("DataFileFolder"));
System.out.println("DBName: " + context.getProperty("DBName"));
System.out.println("JobName: " + context.getProperty("JobName"));
System.out.println("JobRunID: " + context.getProperty("JobRunID"));
4) Run "ContextLoad" job. Here is an output:
Starting job ContextLoad at 12:42 02/03/2012.
connecting to socket on port 4062
connected
tContextLoad_1 set key "DataFileFolder" with value "D:/tmp/Test/"
tContextLoad_1 set key "DBName" with value "testing_derek"
tContextLoad_1 set key "JobName" with value "S301_test_context"
BEFORE the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testing_derek
JobName: S301_test_context
JobRunID: 1
BEFORE the SUBJOB and after changing JobRunID context:
JobRunID: 1
INSIDE of the SUBJOB:
Data File Folder: null
DBName: null
JobName: S301_test_context
JobRunID: 2
disconnected
Job ContextLoad ended at 12:42 02/03/2012.
Issue:
As you can see there is an issues:
1) I expect that all context variables loaded by tContextLoad component would be transferred to SubJob, but only one is transferred "JobName". Why other context variables are not transferred over?
Thank you in advance,
Elena

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Mon, 05 Mar 2012 05:45:06 GMT

Hi Elena
Please show me the schema of tPostgresqlInput and settings of tRunjob.
I guess you didn't configure it with tContextLoad correctly.
Regards,
Pedro

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Thu, 08 Mar 2012 03:30:12 GMT

Hi Pedro,
Please check your e-mail for images - I was not able to upload images via Forum
thanks again,
Elena

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Sat, 10 Mar 2012 03:26:33 GMT

Hi Pedro,
Did you have a chance to look at this issue?
Any help would be greately appreciated.
Thank you in advance,
elena

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Mon, 12 Mar 2012 02:10:06 GMT

Hi Elena
Sorry for my delay to answer.
Are you sure you have added context "DataFileFolder" and "DBName" in DisplayContext_vars?
From the images which you sent, I can't figure out the reason.
You'd better export items of these two jobs and sent email to me.
I will test it for you.
Regards,
Pedro

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Mon, 12 Mar 2012 17:41:04 GMT

Hi Pedro,
I think the Context vars are setup correctly - please see below the Job execution log:
As you can see tContextLoad_1 component is printing out the variables and values, including "DBName" and "DBFileFolder".
Than Java code is able to access these 2 variables in tJava_1 component - they are displayed in job log after title "BEFORE the SUBJOB:".
However, when SubJob is trying to access these variables - they are Null - they are displayed from SubJob in job log after title "INSIDE of the SUBJOB:".
I have exported the Job and all dependencies.
Please check your e-mail box for the file.
Thank you for all your time and support,
Elena

Job Execution Log:
Starting job ContextLoad at 10:20 12/03/2012.
connecting to socket on port 3626
connected
tContextLoad_1 set key "DataFileServer" with value "yvr-etl-23-01"
tContextLoad_1 set key "DataFileFolder" with value "D:/tmp/Test/"
tContextLoad_1 set key "DataFileETLServer" with value "yvr-etl-23-01"
tContextLoad_1 set key "DataFileETLFolder" with value "D:/tmp/Test/"
tContextLoad_1 set key "DBServerHost" with value "localhost"
tContextLoad_1 set key "DBServerPort" with value "5432"
tContextLoad_1 set key "DBName" with value "testing_derek"
tContextLoad_1 set key "FiscalPeriod" with value "Y"
tContextLoad_1 set key "OrgHierarchy" with value "N"
tContextLoad_1 set key "CompensationMarketData" with value "N"
tContextLoad_1 set key "CompensationPayLevel" with value "N"
tContextLoad_1 set key "OrgHierarchyInd" with value "0"
tContextLoad_1 set key "DBSystemServerHost" with value "localhost"
tContextLoad_1 set key "DBSystemServerPort" with value "5432"
tContextLoad_1 set key "DBSystemName" with value "_base_etl"
tContextLoad_1 set key "DBPassword" with value "reisiv123"
tContextLoad_1 set key "DBUserName" with value "talenduser"
tContextLoad_1 set key "DBSystemUserName" with value "talenduser"
tContextLoad_1 set key "DBSystemPassword" with value "reisiv123"
tContextLoad_1 set key "JobName" with value "S301_test_context"
tContextLoad_1 set key "CustomerID" with value "1"
tContextLoad_1 set key "Status" with value "F"
tContextLoad_1 set key "Attr1" with value "200901"
tContextLoad_1 set key "Attr2" with value "Employee_200901.txt"
tContextLoad_1 set key "Attr3" with value """"
tContextLoad_1 set key "Attr4" with value """"
tContextLoad_1 set key "Attr5" with value """"
BEFORE the SUBJOB:
Data File Folder: D:/tmp/Test/
DBName: testing_derek
JobName: S301_test_context
JobRunID: 1
BEFORE the SUBJOB and after changing JobRunID context:
JobRunID: 1
INSIDE of the SUBJOB:
Data File Folder: null
DBName: null
JobName: S301_test_context
JobRunID: 2
disconnected
Job ContextLoad ended at 10:20 12/03/2012.

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Tue, 13 Mar 2012 02:21:23 GMT

Hi Elena
I import your jobs and change the postgresql parameters into my local DB.
It works fine.
Which version of TOS do you use?
Maybe you encounter a bug.
Regards,
Pedro

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Tue, 13 Mar 2012 21:18:04 GMT

hi Pedro,
Our version is 5.0.1.
Which version do you use?
Maybe I should upload it and try to reproduce the issue?
Please advise.
Thanks again,
Elena

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Wed, 14 Mar 2012 03:30:57 GMT

Hi Elena
My TOS is also 5.0.1.
I delved into the code in your job "ContextLoad" and found some useless methods in it.

public void connectionStatsLogsProcess()

This must be due to code generation issue.
Now copy all these components into a new job.
I guess this issue will be fixed.
Regards,
Pedro

Re: tRunJob does not transfer Context variables when "Use dynamic job" sel

Anonymous — Wed, 28 Mar 2012 22:46:05 GMT

Hi Pedro,
Thank you for your detailed review of the code.
I tried your suggestion.
Unfortunately, it did not work for me.
New Job has same issue - it is inconsistent with transferring Context variables to SubJob when Dynamic job is selected.
Please let me know if you have another suggestions.
Thank you in advance,
Elena