Transmit custom object to a child job with dynamic job option

[Anonymous]

Hi,

we have an issue trying to transmit a custom object to a child job. 

We have a custom java class "MyClass" with some variables.

In the parent job we initialize a context variable "myObject" of type MyClass and its variables. Then, in the tRunJob component, we transmit this context variable to the child job.

The child job simply prints MyClass variables.

If we simply call the child job, it runs correctly.

If we check the "Use dynamic job" option and call the child job through a context variable "jobName" we get a java.lang.ClassCastException.

This is the log:

Starting job my_parent_job at 11:55 24/09/2018.

[statistics] connecting to socket on port 4082
[statistics] connected

Exception in component tRunJob_1
java.lang.RuntimeException: Child job returns 1. It doesn't terminate normally.
Exception in component tRowGenerator_1
java.lang.ClassCastException: java.lang.String cannot be cast to routines.MyClass
at janus_olap_etl.my_child_job_0_1.my_child_job$1tRowGenerator_1Randomizer.getRandommyFirstVar(my_child_job.java:551)
at janus_olap_etl.my_child_job_0_1.my_child_job.tRowGenerator_1Process(my_child_job.java:558)
at janus_olap_etl.my_child_job_0_1.my_child_job.runJobInTOS(my_child_job.java:1815)
at janus_olap_etl.my_child_job_0_1.my_child_job.main(my_child_job.java:1675)

at janus_olap_etl.my_parent_job_0_1.my_parent_job.tRunJob_1Process(my_parent_job.java:873)
at janus_olap_etl.my_parent_job_0_1.my_parent_job.runJobInTOS(my_parent_job.java:1853)
at janus_olap_etl.my_parent_job_0_1.my_parent_job.main(my_parent_job.java:1689)

[statistics] disconnected
Job my_parent_job ended at 11:55 24/09/2018. [exit code=1]

Have we made some mistakes or is this a bug?

We are using Talend Open Studio for Data Integration 6.3.0. We also tried to upgrade to 7.0.1 but we didn't see any difference.

Thanks

[Anonymous]

This may be a bug. It looks like Talend is receiving the Object as a String. Can you try casting to an Object first and then casting to your Class. That *might* work.

[Anonymous]

Thank you for your suggestion, but it doesn't work. There is no difference in log.

[Anonymous · Accepted solution]

This interested me, so I tried it out. I think this is caused by the fact that Dynamic tRunJobs do not run in the same process as the parent. An independent process is used. What Talend seem to do is implicitly cast the contexts to Strings and transfer them that way. Unfortunately that is no good for Objects of unknown classes. But there is a way you can do this. I have a workaround for you.

First, create a new routine like below. This is a routine used to serialize your Objects to Strings....

package routines;

import java.util.*;
import java.io.*;

/** 
 * Usage sample serializing SomeClass instance 
 */
public class Serializer {

    /** Read the object from Base64 string. */
   public static Object fromString( String s ) throws IOException ,
                                                       ClassNotFoundException {
        byte [] data = Base64.getDecoder().decode( s );
        ObjectInputStream ois = new ObjectInputStream( 
                                        new ByteArrayInputStream(  data ) );
        Object o  = ois.readObject();
        ois.close();
        return o;
   }

    /** Write the object to a Base64 string. */
    public static String toString( Serializable o ) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream( baos );
        oos.writeObject( o );
        oos.close();
        return Base64.getEncoder().encodeToString(baos.toByteArray()); 
    }
}

 You will need to subtly change your Class as well. You need it to implement Serializable. I have done this to a test Class I created below.....

package routines;

import java.io.Serializable;


public class DemoClass implements Serializable {


	public DemoClass(String dataString, int dataInt){
		this.dataString = dataString;
		this.dataInt = dataInt;
	}
	
	public String dataString;
	public int dataInt;
		
}

Now, instead of setting your context variable to Object, set it to String (in all jobs). Now when you want to pass your Object to the child job, serialize it to a String like below....

routines.DemoClass dc = new routines.DemoClass("HelloWorld", 12);
context.MyObject = routines.Serializer.toString(dc);

When you receive your Object as a String in the child job, you can get access to it like below...

routines.DemoClass dc = ((routines.DemoClass)routines.Serializer.fromString(context.MyObject));

System.out.println(dc.dataString);

It is a bit frustrating having to do this, but hopefully you can see that it doesn't take too much work to get around it.

[Anonymous]

Thanks for your help, it works!

It requires a bit of work because we have many child jobs, but at least we have a solution.

Thank you very much