How to run Big Data Spark Job using Talend, getting error when trying to run

Anonymous — Sat, 16 Nov 2024 05:48:30 GMT

Hi,

I am not able to Run Big Data Spark Job,getting below mentioned error,Please suggest.

Error:-

org.talend.designer.runprocess.ProcessorException: Job compile errors
At least job "Test" has a compile errors, please fix and export again.
Error Line: 1108
Detail Message: The method addSparkListener(SparkListenerInterface) in the type SparkContext is not applicable for the arguments (Test.TalendRuntimeSparkListener)
There may be some other errors caused by JVM compatibility. Make sure your JVM setup is similar to the studio.
at org.talend.designer.runprocess.JobErrorsChecker.checkLastGenerationHasCompilationError(JobErrorsChecker.java:338)
at org.talend.designer.runprocess.DefaultRunProcessService.checkLastGenerationHasCompilationError(DefaultRunProcessService.java:464)
at org.talend.designer.runprocess.RunProcessService.checkLastGenerationHasCompilationError(RunProcessService.java:316)
at org.talend.designer.runprocess.ProcessorUtilities.generateBuildInfo(ProcessorUtilities.java:812)
at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:586)
at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:1736)
at org.talend.designer.runprocess.RunProcessContext$1.run(RunProcessContext.java:582)
at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:466)
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:374)
at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:527)
at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:284)
at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1190)
at org.talend.designer.runprocess.RunProcessContext.exec(RunProcessContext.java:534)
at org.talend.designer.runprocess.remote.RunRemoteProcessContext.exec(RunRemoteProcessContext.java:84)
at org.talend.designer.runprocess.ui.ProcessComposite.exec(ProcessComposite.java:1401)
at org.talend.designer.runprocess.ui.ProcessComposite.addInHistoryRunningList(ProcessComposite.java:1352)
at org.talend.designer.runprocess.ui.ProcessComposite.execButtonPressed(ProcessComposite.java:930)
at org.talend.designer.runprocess.ui.ProcessComposite.execRun(ProcessComposite.java:680)
at org.talend.designer.runprocess.ui.ProcessComposite$1.widgetSelected(ProcessComposite.java:396)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.talend.rcp.intro.Application.start(Application.java:265)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
at org.eclipse.equinox.launcher.Main.run(Main.java:1465)

Thank You!!

Mahinder

Re: How to run Big Data Spark Job using Talend, getting error when trying to run

Anonymous — Mon, 27 May 2019 09:10:09 GMT

Hello,

When you get some compile errors, could you please check your "Code" tab in your job to see if there will be your compile error highlighted in red line?

Best regards

Sabrina

topic How to run Big Data Spark Job using Talend, getting error when trying to run in Talend Studio

How to run Big Data Spark Job using Talend, getting error when trying to run

Re: How to run Big Data Spark Job using Talend, getting error when trying to run