topic Re: contextStr not passed to subjob? in Talend Studio

contextStr not passed to subjob?

Anonymous — Wed, 27 Feb 2013 17:37:26 GMT

I'm basing some decisions in my jobs off the context group which I'm getting from contextStr. But when I create a parent/subjob arrangement and pass the context to the subjob, the contextStr does not change. It appears the context is being set, but the variable contextStr is not.
I have created a context with 2 context variables, Environment and testServer. I have 4 context groups, Dev, Test, QA, and Prod. The default is Dev. The context variables in each group are set to indicate the group (in this example).
The jobs for this test are simply a tPrejob triggering a tJava which contains the System.out.println statements. The tRunJob in the parent has the Transmit whole context box checked.
Here is my output when I select Test for the context (default is Dev):
Starting job ParentJob at 12:33 27/02/2013.
connecting to socket on port 3623
connected
In Parent Job.
Using Test context (contextStr)
Environment = Test
Server = testServer
In Child Job.
Using Dev context (contextStr)
Environment = Test
Server = testServer
disconnected
As you can see the parent is using the Test context and the context variables in the child show the correct context, but the contestStr in the child is not correct. It is showing the default context.
I am using Talend Enterprise Data Integration Team edition 5.1.1
Job ParentJob ended at 12:33 27/02/2013.

Re: contextStr not passed to subjob?

Anonymous — Mon, 11 Mar 2013 14:16:52 GMT

hi,
verify that you've got select the correct context (environment) in you trunJob Component.
I use test & dev context env. for the example.
depend on which context I pass to the child job, I've got different "answer".
hope it helps.
regards
laurent
... and be sure it's not a waste of time by using Talend Forum ... community & talend team are doing a real great job
The contextStr is not a documented "global var " and few of us use it, I guess.

Re: contextStr not passed to subjob?

Anonymous — Mon, 11 Mar 2013 14:29:12 GMT

I'm not sure what you are doing. It looks like you are setting the Context dropdown in the tRunJob component. Yes, that works, but that's not what we need.
We need the context of the parent (the job holding the tRunJob) to be sent to the subjob. The context of the parent is dynamic, only being set at run time. Our context variables are being sent down to the subjob based on this dynamic context. But the Talend global? variable contextStr is not.
Parent
context at run time = dev
contextStr = 'dev'
Context field on tRunJob = dev
Child
context at run time = dev
contextStr = 'dev'
Different run:
Parent
context at run time = test
contextStr = 'test'
Context field on tRunJob = dev
Child
context at run time = test
contextStr = 'dev' <------

Re: contextStr not passed to subjob?

Anonymous — Mon, 11 Mar 2013 15:37:37 GMT

here's the way Talend manage the contextStr :

// call job/subjob with an existing context, like:
			// --context=production. if without this parameter, there will use
			// the default context instead.
			java.io.InputStream inContext = contextPere.class.getClassLoader()
					.getResourceAsStream(
							"itf_sap/contextpere_0_1/contexts/" + contextStr
									+ ".properties");
			if (isDefaultContext && inContext == null) {
			} else {
				if (inContext != null) {
					// defaultProps is in order to keep the original context
					// value
					defaultProps.load(inContext);
					inContext.close();
					context = new ContextProperties(defaultProps);
				} else {
					// print info and job continue to run, for case:
					// context_param is not empty.
					System.err.println("Could not find the context "
							+ contextStr);
				}
				if (!context_param.isEmpty()) {
					context.putAll(context_param);
				}

but never mind.
If you run job from Studio, child job use the context specify in the "box" or default if not.
if you export job, you can tell wich context env to use (@screenshoot)
if you're using export via TAC you can also 'force' father&child to use the same context param.
But the contextStr is "just a flag" to tell which value in context group to use & of course father and child have to use the same env.

regards
laurent

Re: contextStr not passed to subjob?

Anonymous — Mon, 11 Mar 2013 16:23:29 GMT

Not completely true.

If I'm in the Studio and the checkbox "Transmit whole context" is checked on the tRunJob, the subjob will use the context of the parent (job holding the tRunJob) regardless of what is selected in the Context field of tRunJob. The listing in my first post show this.
I don't understand your last comment. Is contextStr an environment variable? It has no relationship to the context used by the child job. In my tests, when the "Transmit whole context" is checked, the child job uses the parent context. The contextStr in the child job does not match the context actually being used. I think that is a bug. If the job is using context 'A', contextStr for that job should be 'A'.

Re: contextStr not passed to subjob?

Anonymous — Mon, 11 Mar 2013 21:33:22 GMT

the fact is that you can run different context between father & child (even if it's a "non-sense", but it could be).
Create a childJob and a father jog & left Default context.
Create 2 context variable (for ex. : firstname & lastname) in each of them.
in father job put some value (ex. firstname = david & lastname = bowie)
in childJob lest null value
write values of context variable & contextStr in childJob in console

System.out.println("contextStr : " + contextStr);
System.out.println("firstname : " + context.firstname);
System.out.println("lastname : " + context.lastname);

run childJob :

Starting job f28400 at 21:55 11/03/2013.

 connecting to socket on port 3543
 connected
contextStr : Default
firstname : null
lastname : null
 disconnected
Job f28400 ended at 21:55 11/03/2013.

run father Job ... same result
check "transmit whole context"

Starting job father_f28400 at 21:58 11/03/2013.

 connecting to socket on port 3336
 connected
contextStr : Default
firstname : david
lastname : bowie
 disconnected
Job father_f28400 ended at 21:58 11/03/2013.

Now add an new context environnment ; let's say "test" in ChilJob (@see screenshoot)
you can run childJob with 2 different environmentt (Default & test)
Run childJob with those 2 differents env. => change the value of contextStr

 connecting to socket on port 3708
 connected
contextStr : Default
firstname : null
lastname : null
 disconnected

 connecting to socket on port 3888
 connected
contextStr : test
firstname : null
lastname : null
 disconnected

Back to father that still have only a Default context with values "david" "bowie".
But you can select the context for the childJob (Default or test ) in component View.
See what happen with and without "transmit whole context"
With transmition , overwrite the "null value" of childJob, without take the value of the child depending of the "context passed in component view" . (for now , always "null" result.
To check , give different value in child job for firstname & lastname :
Default : kirk hammet
test : nina hagen

First run Child

Starting job f28400 at 22:21 11/03/2013.

 connecting to socket on port 3718
 connected
contextStr : Default
firstname : kirk
lastname : hammet
 disconnected
Job f28400 ended at 22:21 11/03/2013.

Starting job f28400 at 22:22 11/03/2013.

 connecting to socket on port 3829
 connected
contextStr : test
firstname : nina
lastname : hagen
 disconnected
Job f28400 ended at 22:22 11/03/2013.

Again back to father & run with or without "transmit whole context " and with the 2 differents child context in component view.
with transmit ....

Starting job father_f28400 at 22:23 11/03/2013.

 connecting to socket on port 3455
 connected
contextStr : test
firstname : david
lastname : bowie
 disconnected
Job father_f28400 ended at 22:23 11/03/2013.

Even if you change the context in component view ... overwrite value
without ...

Starting job father_f28400 at 22:25 11/03/2013.

 connecting to socket on port 3655
 connected
contextStr : Default
firstname : kirk
lastname : hammet
 disconnected
Job father_f28400 ended at 22:25 11/03/2013.

Starting job father_f28400 at 22:25 11/03/2013.

 connecting to socket on port 3791
 connected
contextStr : test
firstname : nina
lastname : hagen
 disconnected
Job father_f28400 ended at 22:25 11/03/2013.

.. depending which context is passed !
And now for the fun , add a second context environment for the father ... and run all the different possibilities !
hope you see differences now
regards
laurent

Re: contextStr not passed to subjob?

Anonymous — Tue, 12 Mar 2013 13:53:22 GMT

@kzone, thank you for your detailed post. But it misses the point. In your 8th code widow you show a father with one context environment (Default) and context variables first_name = David, last_name = Bowie.
But the contextStr in the child says test which is NOT the father context which was supposed be transmitted down to the child.
Similarly, the context of David and Bowie in the child is not the test context of the child, even though the contextStr says it is.
In short, the contextStr value does not indicate the context variables the child is actually using. In your example in code window 8, the child is using the Default context values from the father job, but is stating, through the contextStr, it is using context test.
There is a problem in that the child can contain context variables not found in the father. What should the system to then? There's nothing to transmit down. I guess the child should default to whatever context was specified by the tRunJob component.
Never the less, the father job is the main job. This is the one where a context environment is defined at run time. Are we running in Dev, Test, QA, or Production? It's more important to know what runtime context was defined and being used for the whole job than to worry about the static default child context. I thought contextStr should show the context being used. Instead it shows the default or preset context of the current job. There is no indicator that shows the context of a job is being overwritten by a parent job.
I solved my problem by creating my own context variable called environment, transmitting it down to the child, and ignoring contextStr.

Re: contextStr not passed to subjob?

Anonymous — Tue, 12 Mar 2013 16:57:24 GMT

But the contextStr in the child says test which is NOT the father context which was supposed be transmitted down to the child

That's the point.
I think there is a good reason why "Talend doesn't documented" this variable.
It's not the way we've to manage context between father/child.

There is no indicator that shows the context of a job is being overwritten by a parent job.

the checked box option "transmit whole context" is "the indicator".
I use to left child job context value with null value (if it's possible) to "be sure" that father is the only one which have the "knownledge" about manage context (load value from db or file & pass values that child have to work with).
You can decide that child jobs don't have to know about environment context.(dev or prod for ex.)
ChildJob just have to declare context value is going work with.
But father job is the "boss" and just give only the value the child need and tell which environment is used.
run father in DEV context env.

 connecting to socket on port 3946
 connected
localhost
 disconnected

run father in PROD context env.

 connecting to socket on port 4064
 connected
funny.bunny.com
 disconnected

happy you find a workaround, even if it's seems a little "tricky".
regards
laurent

Re: contextStr not passed to subjob?

Anonymous — Tue, 12 Mar 2013 18:26:20 GMT

By indicator I meant something in the child job so it would know what it's running. As I stated in my original post, we have logic in our child jobs that depend on what context the parent job is running. In our design there is only one set of context variables (apparently duplicated in each child). But the context environment or group or whatever it's called that sets the group is selected at run time. I expected some built in context string or something to tell the child what the runtime context was.
I'm not sure how we found the contextStr variable, but it makes sense why it's not more prominent. I was assuming (erroneously) that when a context 'set' was set at runtime, it would apply to all subjobs of the job that was kicked off. It seems silly to run the parent in 'production' and the child in 'development'. But I now see that a child may not even have the same context variables. I also thought the child would inherit the context variables of the parent.
I see the preferred method of keeping the child in sync with the father is to explicitly transmit the context down, including the context name it that is needed.
Thanks for your help and insight.

Re: contextStr not passed to subjob?

Anonymous — Tue, 12 Mar 2013 20:38:44 GMT

I was assuming (erroneously) that when a context 'set' was set at runtime, it would apply to all subjobs of the job that was kicked off

you got it (@screenshot)
regards
laurent

Re: contextStr not passed to subjob?

_AnonymousUser — Sat, 30 Mar 2013 13:02:52 GMT

You can decide that child jobs don't have to know about environment context.(dev or prod for ex.)
ChildJob just have to declare context value is going work with.
But father job is the "boss" and just give only the value the child need and tell which environment is used.