Add Dependencies for Routine

[ankushdeshpande]

Hello All,

I have written a routine to run a Java code to read a file and store as RDD. i have added few Spark & Scala dependencies using "Edit Routine Libraries"

When i run the component tJava with call to routine, it fails with classNotFound Exception. once i add that particular Dependency, the error goes away but job fails for another ClassNotFound error.

I already have all the JARS present in "\.m2\repository" path, so not sure how a routine can use all existing JARS.

Routine :

package routines;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;

public class file_to_rdd {

public static void main(String filepath) {
// configure spark
SparkConf sparkConf = new SparkConf();

// start a spark context
JavaSparkContext sc = new JavaSparkContext(sparkConf);

// provide path to input text file
String path = filepath;

// read text file to RDD
JavaRDD<String> lines = sc.textFile(path);

// collect RDD for printing
for(String line:lines.collect()){
System.out.println(line);
}
}

}

[Anonymous]

Hello,

In studio, right click on routine and select 'Edit routine libraries' option to add external libraries to your routine.

With talend subscription solution, you can open the configuration page in TAC and define the User libraries parameters to specify a repository URL for storing all libraries. In this way, your  jar files should be loaded automatically to Nexus after you configure the parameters and the dependency libraries are exported also when deploy the job to remote Job server.

Let us know if it helps.

Best regards

Sabrina

[Anonymous]

Hi,

    An example about how to add jars to routines can be referred from below link.

https://community.talend.com/t5/Design-and-Development/Talend-and-Amazon-Comprehend-Integration/ta-p...

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved