Converting an arbitrary flow in to a Java map that is passed to JAR

[Anonymous]

Within a tJavaRow or tJavaFlex, I would like to convert the incoming flow into a Java map (i.e. HashMap) that I can then pass to a method on an external JAR. The map would be of the form:
HashMap<String, String> hm
And for each column in the flow (columns are not not known beforehand) hm.name would have the colulmn name and hm.value would have the column value.
Do I need to leverage Java reflection to accomplish this, or is there a more straightforward, Talend-centric way to do so?

[Anonymous]

Hi Rob
I think Talend doesn't allow you to catch the schema names directly. You can have a workaround for this like
tOracleInput----row1---->tJavaRow
in tJavaRow
row1Struct mapEntry=new row1Struct();
routines.<hashmapdeclaredClass>.put("colname1",row.colname1)
here the hashmap should be static.
this will be more of a static solution where the col names has to be constant.
Hope it helps you.

[Anonymous]

Thanks Lijo. The problem with this approach, of course, is that it requires a priori knowledge of the schema columns. What I was trying to create is a generic routine that, given a flow like row1, know how to create a hashmap corresponding to the column/value pairs, without having to know the column names. I solved the problem using Java reflection as shown below.
I created a routine as follows:
package routines;
import java.lang.reflect.*;
import java.util.*;
public class SPIxGenWrapper {
public static Map<String, String> flowToMap(Class flowClass, Object flow) {
Map<String, String> retMap = new HashMap<String, String>();

try {
Field[] fields = flowClass.getFields();
List<Field> fl = Arrays.asList(fields);
for(Field f : fl) {
retMap.put(f.getName(), f.get(flow).toString());
}
} catch(Exception e) {System.out.println(e);}

return (retMap);
}
}
I can now invoke this from within, for example, a tJavaRow as follows:
java.lang.Class c = input_row.getClass();
SPIxGenWrapper.flowToMap(c, input_row);
Of course Talend replaces "input_row" with the actual class name in its code.
This approach seems to work reliably so far, and is tremendously useful anytime you want to invoke a method (in a routine, external library, etc.) or retain a flow for later use (of course you can use tBufferInput/tBufferOutput for the latter purpose). Note that it only works for String columns (all I need for now), but I believe it could be enhanced to treat types more generally.

[Anonymous]

Hi Rob,
Thanks, It looks nice. I will also try in my project .

[Anonymous]

You could also do this with a Dynamic column. Define the database schema as a single column with type Dynamic, and that column will, at run-time, contain all your real columns with their names and type information, and of course the values. Using the Dynamic API, you can extract those values and feed them into your HashMap.
(Edit: Dynamic columns are only available in the subscription version though. I saw too late that this is the Open Studio forum.)

[bluishjoe]

@roboaks1, if I ever meet you in the future I'll be happy to invite you for a drink!
I used your solution to load input_row's columns to the global vars and works perfectly!

package routines;
import java.lang.reflect.*;
import java.util.*;
public class JobUtils {
    public static void inputCols2globalVars(Class flowClass, Object flow, Map<String, Object> globalMap) 
    		throws IllegalArgumentException, IllegalAccessException{
    	Field[] fields = flowClass.getFields();
        List<Field> fl = Arrays.asList(fields);
        for(Field f : fl)
            globalMap.put(f.getName(), f.get(flow));
    }
}

Invoked with:

JobUtils.inputCols2globalVars(input_row.getClass(), input_row, globalMap);

Thanks!

[Anonymous]

You're very welcome bluish!
This solution turned out to be profoundly important to the technical approach we were pursuing. In particular, the JAR we created, SPIxGen, wraps XMLBeans generated classes (XMLBeans is an Apache XML data binding tool that is essentially 100% XSD standard compliant) and can generate arbitrarily complex XML based on an XSD. Because of the way we defined the interface, SPIxGen is completely generic and does not need to understand the XML it is generating. However, it does require that the element/attribute be passed in a HashMap, and we wanted to accomplish the HasMap creation automatically.
We went with the SPIxGen approach because our research convinced us it would be very difficult to achieve the same result using the native Talend XML components. Talend has worked marvelously well for us overall, but this is one area where we needed to take a Java-based approach.