Hello,
I have created a job in talend to connect to a Spark 1.6.0 Cluster.
When running the job from Talend an error is thrown.
Configuration:
Spakr 1.6 , spark-mode : standalone, distrubition : custom
Any help on this would be highly appreciated since this issue is a real showstopper !!
java.lang.NoClassDefFoundError: org/apache/spark/scheduler/SparkListener
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.scheduler.SparkListener
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
Error: A JNI error has occurred, please check your installation and try again
Hi, Could you please indicate the build version you are using? Are you using jdk 1.8? Have you ever executed your other jobs in your Talend studio successfully? Screenshots of job setting will be preferred. Best regards Sabrina
I think you need just to configure the job to use Spark. Go to "Run" tab -> Spark configuration -> load the properties from the repository ( that i presume you created under metadata as hadoop cluster )
