Talend Spark job compilation errors at runtime even though there are no compilation error

Anonymous — Sat, 16 Nov 2024 07:42:30 GMT

We ran into an issue while running StoreDB, ProductDB and STLD Talend spark jobs to convert XML files to CSV. Though code doesn't show any compilation error, while running the jobs are failing saying there is a compilation error.

Re: Talend Spark job compilation errors at runtime even though there are no compilation error

Anonymous — Thu, 06 Sep 2018 03:42:22 GMT

Hello,

Could you please post the compilation errors you got on forum?

In addition, if you don't mind, could you please open the error log generated automatically by the Studio to see whether you have error stack trace there; if you do, we will appreciate it a lot if you can paste it here.

Best regards

Sabrina

Re: Talend Spark job compilation errors at runtime even though there are no compilation error

Anonymous — Thu, 06 Sep 2018 16:36:39 GMT

Here is the log"

org.talend.designer.runprocess.ProcessorException: Job compile errors

At least job "XML_CSV_POS_STOREDB" has a compile errors, please fix and export again.

Error Line: 12

Detail Message: The method parse(String) in the type Schema.Parser is not applicable for the arguments (String, String, String)

There may be some other errors caused by JVM compatibility. Make sure your JVM setup is similar to the studio.

at org.talend.designer.runprocess.JobErrorsChecker.checkLastGenerationHasCompilationError(JobErrorsChecker.java:328)

at org.talend.designer.runprocess.DefaultRunProcessService.checkLastGenerationHasCompilationError(DefaultRunProcessService.java:380)

at org.talend.designer.runprocess.RunProcessService.checkLastGenerationHasCompilationError(RunProcessService.java:292)

at org.talend.designer.runprocess.ProcessorUtilities.generateBuildInfo(ProcessorUtilities.java:656)

at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:545)

at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:1506)

at org.talend.designer.runprocess.RunProcessContext$1.run(RunProcessContext.java:577)

at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:466)

at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:374)

at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:527)

at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:284)

at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1190)

at org.talend.designer.runprocess.RunProcessContext.exec(RunProcessContext.java:532)

at org.talend.designer.runprocess.remote.RunRemoteProcessContext.exec(RunRemoteProcessContext.java:86)

at org.talend.designer.runprocess.ui.ProcessComposite.exec(ProcessComposite.java:1450)

at org.talend.designer.runprocess.ui.ProcessComposite.addInHistoryRunningList(ProcessComposite.java:1401)

at org.talend.designer.runprocess.ui.ProcessComposite.execButtonPressed(ProcessComposite.java:935)

at org.talend.designer.runprocess.ui.ProcessComposite.execRun(ProcessComposite.java:674)

at org.talend.designer.runprocess.ui.ProcessComposite$1.widgetSelected(ProcessComposite.java:395)

at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)

at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)

at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)

at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)

at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)

Re: Talend Spark job compilation errors at runtime even though there are no compilation error

Anonymous — Fri, 07 Sep 2018 04:38:01 GMT

Hello,

In your spark job, are you using EMR version 5.5.0?

Here is a known issue on talend bug tracker:https://jira.talendforge.org/browse/TBD-5390

If not, could you please post your spark job design screenshots on forum? Pleae mask your sensitive data.

Best regards

Sabrina

topic Re: Talend Spark job compilation errors at runtime even though there are no compilation error in Talend Studio

Talend Spark job compilation errors at runtime even though there are no compilation error

Re: Talend Spark job compilation errors at runtime even though there are no compilation error

Re: Talend Spark job compilation errors at runtime even though there are no compilation error

Re: Talend Spark job compilation errors at runtime even though there are no compilation error