topic Re: [resolved] retrieve globalMap to row in Talend Studio

[resolved] retrieve globalMap to row

Anonymous — Sat, 16 Nov 2024 12:55:08 GMT

Hi,
row1 ---(main) ---> tjavaflex ---> tMySQLOutput
tJavaFlex main, I insert rows to a global HashSet.
tJavaFlex end, I export the HashSet

globalMap.put("specialPairs", specialPairs);

How can I use the content of the globalMap as row input in the next component, such as tjavarow or tMysqlOutput ?
PS.
I am able to iterate and System.out.println() the content of the globalMap in a tjava with

HashSet unique = (HashSet)globalMap.get("specialPairs");

but it's just console output which I can't use in any other components.

Re: [resolved] retrieve globalMap to row

Anonymous — Tue, 17 May 2011 19:12:46 GMT

In code, a subjob is a set of nested loops. Each component consists of BEGIN, MAIN, and END code. When components are connected within a subjob, these code sections are composed in the following manner:
input--row-->tJavaFlex--row-->Output
Output_1 START {
tJavaFlex_1 START {
Input_1 START {
Input_1 MAIN { }
tJavaFlex_1 MAIN { }
Output_1 MAIN { }
} Input_1 END
} tJavaFlex_1 END
} Output_1 END

so to insert data into rows with a tJavaFlex, you must do it in the MAIN section of the code.

Re: [resolved] retrieve globalMap to row

Anonymous — Tue, 17 May 2011 20:35:21 GMT

Thanks John,
the problem I have with just forwarding the rows from MAIN part is that I add each row to a unique HashSet to discard duplicates (based on a custom hashCode() and equals() method of my java object).
Thus, I have to finish adding all rows to the global map before I can iterate over them again.
One workaround I found was, that I write the global map into a CSV file (custom java code again) and read it with tFileInputDelimted...not very elegant, was hoping there is a way to iterate in-memory over the globalMap and inject it directly as an input row

Re: [resolved] retrieve globalMap to row

Anonymous — Tue, 17 May 2011 20:47:49 GMT

probably the cleanest way to do this is to process the data in one subjob, and then read from your hash in a second subjob:
input-->tJavaFlex
|
onSubjobOK
|
tJavaFlex-->output
in the second tJavaFlex, you can iterate over the globalMap and output rows

Re: [resolved] retrieve globalMap to row

Anonymous — Wed, 18 May 2011 12:08:42 GMT

I see your point, gonna give it a try. Thx

Re: [resolved] retrieve globalMap to row

Anonymous — Wed, 18 May 2011 12:24:11 GMT

John,
I'm missing something...if I put something like

for (int i = 0; i < 5; i ++) {
row15.dc1 = i;
row15.dc2 = i+1;
}

in the MAIN for tjavaflex, it only outputs 1 row. (the last one of the iteration actually)

Re: [resolved] retrieve globalMap to row

Anonymous — Wed, 18 May 2011 12:33:51 GMT

Ok found the solution here:
https://community.talend.com/t5/Design-and-Development/resolved-tJavaFlex-multiple-rows/td-p/90245
basically the loop has to be opened in the START part

for (int i = 0; i < 3; i   ) {

rows go in the MAIN part

row15.dc1 = i;
row15.dc2 = i 1;

and the closing of the loop goes to the END part.