topic Re: Accessing Values from SetGlobalvar and tBufferOutput and Input in Talend Studio

Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Sat, 16 Nov 2024 12:39:50 GMT

Hello,
i have the following problem, which is discussed here several times, but after reading these threads i cannot find a solution.
Description:
I want to share values between jobs.
As i read here, it is not possible to share these values with SetGlobalVar.
Some solutions here works with the tBuffer component.
So I put the values from Global into tBufferOutput.
If i understand it correctly, i can read this buffer in other jobs with tBufferInput, too!.
But i have no luck doing this.
Any idea?
Another solution is to put the values in context.
But how do i copy a globalvar into a context var?
Should i do this with tJava component?
Could i access these values all over my jobs?
Here are my screenshots of the main and childjob.
Hope it helps.
Best wishes
Jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

alevy — Mon, 26 Sep 2011 03:11:04 GMT

You cannot share globalMap between jobs. Context variables can easily be passed to child jobs by checking "Transmit whole context" in tRunJob. But passing values back to a parent job is harder.
Problems with your screenshots are:
1. The row coming out of a tSetGlobalVar is just a duplicate of the row going in to it (if any), not any globalMap variables you might have set in it. You need to use tFixedFlowInput to create a flow of your globalMap variables, which you can then pass to tBufferOutput.
2. To retrieve the values from the buffer in the parent, you should have a row directly from tRunJob. (Don't forget to set the schema.) You then need either tSetGlobalVar or tContextLoad or tJavaRow (as appropriate) to convert that flow back to globalMap or context variables in the parent.
Converting globalMap into context variables (or vice versa) has to be done manually - tJava is fine.
There are indeed many posts about passing data between parent and child jobs and back so I suggest you review them again.

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Mon, 26 Sep 2011 17:06:09 GMT

You cannot share globalMap between jobs. Context variables can easily be passed to child jobs by checking "Transmit whole context" in tRunJob. But passing values back to a parent job is harder.
Problems with your screenshots are:
1. The row coming out of a tSetGlobalVar is just a duplicate of the row going in to it (if any), not any globalMap variables you might have set in it. You need to use tFixedFlowInput to create a flow of your globalMap variables, which you can then pass to tBufferOutput.
2. To retrieve the values from the buffer in the parent, you should have a row directly from tRunJob. (Don't forget to set the schema.) You then need either tSetGlobalVar or tContextLoad or tJavaRow (as appropriate) to convert that flow back to globalMap or context variables in the parent.
Converting globalMap into context variables (or vice versa) has to be done manually - tJava is fine.

Hello,
i tried the following with tJava, but it doesn´t work. Perhaps i should use tJavaRow?
Want to save an input row to context variables and then dump it with tContextDump.
Here is the Code inside tJava. The Input Rowname is hoechstes_datetime.
I think, that the name of the rows are not accessible in java that way.

context.put("dirty_datetime_last",hoechstes_datetime.dirty_datetime_last);
context.put("dirty_datetime_new",hoechstes_datetime.dirty_datetime_new);
context.put("aggregation_last",hoechstes_datetime.aggregation_last);
context.put("integration_last",hoechstes_datetime.integration_last);
context.put("date_from",hoechstes_datetime.date_from);
context.put("date_until",hoechstes_datetime.date_until);
context.put("kw_von",hoechstes_datetime.kw_von);
context.put("kw_bis",hoechstes_datetime.kw_bis);
context.put("tag_von",hoechstes_datetime.tag_von);
context.put("tag_bis",hoechstes_datetime.tag_bis);
context.put("monat_von",hoechstes_datetime.monat_von);
context.put("monat_bis",hoechstes_datetime.monat_bis);
context.put("jahr_von",hoechstes_datetime.jahr_von);
context.put("jahr_bis",hoechstes_datetime.jahr_bis);
context.put("stunde_von",hoechstes_datetime.stunde_von);
context.put("stunde_bis",hoechstes_datetime.stunde_bis);
context.put("anzahl",hoechstes_datetime.anzahl);
Also i tried the suggestion with tJavaRow from shong https://community.talend.com/t5/Archive/resolved-how-to-make-a-context/td-p/176105
context.dirty_datetime_last = input_row.dirty_datetime_last;
context.dirty_datetime_new = input_row.dirty_datetime_new;
context.aggregation_last = input_row.aggregation_last;
context.integration_last = input_row.integration_last;
context.date_from = input_row.date_from;
context.date_until = input_row.date_until;
context.kw_von = input_row.kw_von;
context.kw_bis = input_row.kw_bis;
context.tag_von = input_row.tag_von;
context.tag_bis = input_row.tag_bis;
context.monat_von = input_row.monat_von;
context.monat_bis = input_row.monat_bis;
context.jahr_von = input_row.jahr_von;
context.jahr_bis = input_row.jahr_bis;
context.stunde_von = input_row.stunde_von;
context.stunde_bis = input_row.stunde_bis;
context.anzahl = input_row.anzahl;
But this throws an error, too.
Exception in thread "main" java.lang.Error: java.lang.Error: java.lang.Error: java.lang.Error: Unaufgelöste Kompilierungsprobleme:
dirty_datetime_last kann nicht aufgelöst werden oder ist kein Feld
dirty_datetime_new kann nicht aufgelöst werden oder ist kein Feld
aggregation_last kann nicht aufgelöst werden oder ist kein Feld
integration_last kann nicht aufgelöst werden oder ist kein Feld
date_from kann nicht aufgelöst werden oder ist kein Feld
date_until kann nicht aufgelöst werden oder ist kein Feld
kw_von kann nicht aufgelöst werden oder ist kein Feld
kw_bis kann nicht aufgelöst werden oder ist kein Feld
tag_von kann nicht aufgelöst werden oder ist kein Feld
tag_bis kann nicht aufgelöst werden oder ist kein Feld
monat_von kann nicht aufgelöst werden oder ist kein Feld
monat_bis kann nicht aufgelöst werden oder ist kein Feld
jahr_von kann nicht aufgelöst werden oder ist kein Feld
jahr_bis kann nicht aufgelöst werden oder ist kein Feld
stunde_von kann nicht aufgelöst werden oder ist kein Feld
Any suggestions?
Jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Tue, 27 Sep 2011 04:51:52 GMT

Hi
It is a compilation error, have you defined the schema of tJavaRow?
you should use tJavaRow replace tJava if you want to access the columns. I suggest you to use 'onsubjobook' in the job, it makes the job more readable, for example:
tMysqlConnection
|
onsubjobok
|
tFileInputDelimited--main--tContextLoad
|
onsubjobok
|
next processing
Best regards
Shong

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Tue, 27 Sep 2011 09:39:20 GMT

Hi
It is a compilation error, have you defined the schema of tJavaRow?
you should use tJavaRow replace tJava if you want to access the columns.
Best regards
Shong

Hello
thanks for the reply.
yes i have defined a schema for the input of tjavarow, but not for output, because i don´t have any rows to output.
tjavarow should only save the colums from the inputrow to the correspondending context.variables.
here is the screenshot for the schema.
best wishes
jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Tue, 27 Sep 2011 09:57:55 GMT

Hi
It is a compilation error,
Best regards
Shong

Hi,
it´s a bit strange, because the context.variable seems to be not known. Hmmh.
Something wrong with tContextLoad?
The log says, that the variables are loaded and known.
10:55:20|ddFHcP|ddFHcP|ddFHcP|4064|TOS_WETTERDATEN|Copy_of_Uni_Pruefung|_jBqKUOa6EeCETqKxVR9mCQ|1.10|Default|tContextLoad_2|begin||
Warning: Parameter "dirty_datetime_last" is a new parameter of tContextLoad_2
Warning: Parameter "dirty_datetime_new" is a new parameter of tContextLoad_2
Warning: Parameter "aggregation_last" is a new parameter of tContextLoad_2
Warning: Parameter "integration_last" is a new parameter of tContextLoad_2
Warning: Parameter "date_from" is a new parameter of tContextLoad_2
Warning: Parameter "date_until" is a new parameter of tContextLoad_2
Warning: Parameter "kw_von" is a new parameter of tContextLoad_2
Warning: Parameter "kw_bis" is a new parameter of tContextLoad_2
Warning: Parameter "tag_von" is a new parameter of tContextLoad_2
Warning: Parameter "tag_bis" is a new parameter of tContextLoad_2
Warning: Parameter "monat_von" is a new parameter of tContextLoad_2
Warning: Parameter "monat_bis" is a new parameter of tContextLoad_2
Warning: Parameter "jahr_von" is a new parameter of tContextLoad_2
Warning: Parameter "jahr_bis" is a new parameter of tContextLoad_2
Warning: Parameter "stunde_von" is a new parameter of tContextLoad_2
Warning: Parameter "stunde_bis" is a new parameter of tContextLoad_2
Warning: Parameter "anzahl" is a new parameter of tContextLoad_2
Any suggestions?
Jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Tue, 27 Sep 2011 10:18:41 GMT

Hi
It is a compilation error, have you defined the schema of tJavaRow?
you should use tJavaRow replace tJava if you want to access the columns.
Best regards
Shong

jens
Perhaps i find out a solution:
I change the code in the following way
context.put("dirty_datetime_last",input_row.dirty_datetime_last);
context.put("dirty_datetime_new",input_row.dirty_datetime_new);
context.put("aggregation_last",input_row.aggregation_last);
context.put("integration_last",input_row.integration_last);
context.put("date_from",input_row.date_from);
context.put("date_until",input_row.date_until);
context.put("kw_von",input_row.kw_von);
context.put("kw_bis",input_row.kw_bis);
context.put("tag_von",input_row.tag_von);
context.put("tag_bis",input_row.tag_bis);
context.put("monat_von",input_row.monat_von);
context.put("monat_bis",input_row.monat_bis);
context.put("jahr_von",input_row.jahr_von);
context.put("jahr_bis",input_row.jahr_bis);
context.put("stunde_von",input_row.stunde_von);
context.put("stunde_bis",input_row.stunde_bis);
context.put("anzahl",input_row.anzahl);
Now this component seems to work, but the following component tContextDump throws an error:
Exception in component tContextDump_1
java.lang.NullPointerException
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tContextDump_1Process(Copy_of_Uni_Pruefung.java:848)
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tMysqlInput_1Process(Copy_of_Uni_Pruefung.java:5329)
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tWaitForSqlData_1Process(Copy_of_Uni_Pruefung.java:1992)
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tMysqlConnection_1Process(Copy_of_Uni_Pruefung.java:1521)
2011-09-27 11:15:33|KsGnBY|KsGnBY|KsGnBY|4916|TOS_WETTERDATEN|Copy_of_Uni_Pruefung|_jBqKUOa6EeCETqKxVR9mCQ|1.10|Default|tContextDump_1|begin||
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.runJobInTOS(Copy_of_Uni_Pruefung.java:9464)
at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.main(Copy_of_Uni_Pruefung.java:9331)
What could i do wrong here, because the component has no special details...
??

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Wed, 28 Sep 2011 10:22:05 GMT

Hi berufsschule
I can't reproduce the problem, not sure why it throws NPE here. Can you upload a screenshot of job?
Best regards
Shong

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Wed, 28 Sep 2011 10:47:59 GMT

Hi berufsschule
I can't reproduce the problem, not sure why it throws NPE here. Can you upload a screenshot of job?
Best regards
Shong

hello,
here is a screenshot.
hope it helps.
i posted the code from tjavarow some mails above.
Best wishes
Jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Wed, 28 Sep 2011 16:08:27 GMT

Hi
Have you redesign the job as I suggested yesterday?

you should use tJavaRow replace tJava if you want to access the columns. I suggest you to use 'onsubjobook' in the job, it makes the job more readable, for example:
tMysqlConnection
|
onsubjobok
|
tFileInputDelimited--main--tContextLoad
|
onsubjobok
|
next processing
Best regards
Shong

Best regards
Shong

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Wed, 28 Sep 2011 16:31:43 GMT

Hi
Have you redesign the job as I suggested yesterday?
Best regards
Shong

hello,
yes i did.
screenshot below.
i changed the flow into an subjob and changed tjava into tjavarow.
What could i do further?
Any suggestions?
Best wishes
Jens

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Fri, 30 Sep 2011 08:17:31 GMT

Hi
Change the job to:
tMysqlConnection
|
onsubjobok
|
tFileInputDeliited--main--tContextLoad
|
onsubjobok
|
tWaitForDate--itertate-->....
Best regards
Shong

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Anonymous — Fri, 30 Sep 2011 11:06:37 GMT

Hi
Change the job to:
tMysqlConnection
|
onsubjobok
|
tFileInputDeliited--main--tContextLoad
|
onsubjobok
|
tWaitForDate--itertate-->....
Best regards
Shong

Hi,
thanks for the tipp, but same result :-((
here is the code from tjavarow (i changed nothing from the above code)
context.put("dirty_datetime_last",input_row.dirty_datetime_last);
context.put("dirty_datetime_new",input_row.dirty_datetime_new);
context.put("aggregation_last",input_row.aggregation_last);
context.put("integration_last",input_row.integration_last);
context.put("date_from",input_row.date_from);
context.put("date_until",input_row.date_until);
context.put("kw_von",input_row.kw_von);
context.put("kw_bis",input_row.kw_bis);
context.put("tag_von",input_row.tag_von);
context.put("tag_bis",input_row.tag_bis);
context.put("monat_von",input_row.monat_von);
context.put("monat_bis",input_row.monat_bis);
context.put("jahr_von",input_row.jahr_von);
context.put("jahr_bis",input_row.jahr_bis);
context.put("stunde_von",input_row.stunde_von);
context.put("stunde_bis",input_row.stunde_bis);
context.put("anzahl",input_row.anzahl);
context.synchronizeContext();
and here is the screenshot.
Any clues?
Best wishes
Jens