Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable
‎2017-10-23 09:09 AM

tMapRStreamsOutput in Spark Streaming Job

Hi,

 

It says here that it is possible to use tMapRStreamsOutput in an Apache Spark Streaming job.

But when I tried to create a job like this:

0683p000009Lr7q.png

The job build fails with this error message:

org.talend.designer.runprocess.ProcessorException: Failed to generate code.
	at org.talend.designer.runprocess.java.JavaProcessor.generateCode(JavaProcessor.java:572)
	at org.talend.designer.runprocess.maven.MavenJavaProcessor.generateCode(MavenJavaProcessor.java:71)
	at org.talend.designer.runprocess.ProcessorUtilities.generateContextInfo(ProcessorUtilities.java:685)
	at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:511)
	at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:1413)
	at org.talend.designer.runprocess.RunProcessContext$1.run(RunProcessContext.java:576)
	at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:466)
	at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:374)
	at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:527)
	at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:284)
	at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1190)
	at org.talend.designer.runprocess.RunProcessContext.exec(RunProcessContext.java:531)
	at org.talend.designer.runprocess.remote.RunRemoteProcessContext.exec(RunRemoteProcessContext.java:71)
	at org.talend.designer.runprocess.ui.ProcessComposite.exec(ProcessComposite.java:1450)
	at org.talend.designer.runprocess.ui.ProcessComposite.addInHistoryRunningList(ProcessComposite.java:1401)
	at org.talend.designer.runprocess.ui.ProcessComposite.execButtonPressed(ProcessComposite.java:935)
	at org.talend.designer.runprocess.ui.ProcessComposite.execRun(ProcessComposite.java:674)
	at org.talend.designer.runprocess.ui.ProcessComposite$1.widgetSelected(ProcessComposite.java:395)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
	at org.talend.rcp.intro.Application.start(Application.java:255)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
Caused by: org.talend.designer.codegen.exception.CodeGeneratorException: org.eclipse.emf.codegen.jet.JETException: InvocationTargetException in null job
	at org.talend.designer.components.mrprovider.codegen.BigDataCodeGenerator.generateComponentCode(BigDataCodeGenerator.java:185)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentCodeWithHeader(AbstractSparkCodeGenerator.java:133)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:125)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generatesTreeCode(AbstractSparkCodeGenerator.java:160)
	at org.talend.designer.components.sparkprovider.codegen.AbstractSparkCodeGenerator.generateComponentsCode(AbstractSparkCodeGenerator.java:126)
	at org.talend.designer.components.sparkstrprovider.codegen.SparkStrCodeGen.generateProcessCode(SparkStrCodeGen.java:75)
	at org.talend.designer.runprocess.java.JavaProcessor.generateCode(JavaProcessor.java:569)
	... 44 more
Caused by: org.eclipse.emf.codegen.jet.JETException: InvocationTargetException
	at org.eclipse.emf.codegen.jet.JETEmitter.generate(JETEmitter.java:469)
	at org.talend.designer.codegen.config.TalendJetEmitter.generate(TalendJetEmitter.java:684)
	at org.eclipse.emf.codegen.jet.JETEmitter.generate(JETEmitter.java:441)
	at org.eclipse.emf.codegen.jet.JETEmitter.generate(JETEmitter.java:432)
	at org.talend.designer.codegen.proxy.JetProxy.generate(JetProxy.java:54)
	at org.talend.designer.components.mrprovider.codegen.BigDataCodeGenerator.generateComponentCode(BigDataCodeGenerator.java:181)
	... 68 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.emf.codegen.jet.JETEmitter.generate(JETEmitter.java:461)
	... 73 more
Caused by: java.lang.NullPointerException
	at org.talend.designer.common.tsetkeystore.TSetKeystoreUtil.<init>(TSetKeystoreUtil.java:50)
	at org.talend.designer.spark.generator.storage.KafkaSparkStorage.<init>(KafkaSparkStorage.java:49)
	at org.talend.designer.codegen.translators.messaging.maprstreams.TMapRStreamsOutputSparkstreamingcodeJava.generate(TMapRStreamsOutputSparkstreamingcodeJava.java:147)
	... 78 more

But the job builds successfully if I remove the tMapRStreamsOutput component.

Or is it not possible to combine tMapRStreamsInput and tMapRStreamsOutput in a single job?

 

Thank you.

 

Labels (4)
Labels
188 Views
0 Likes
3 Replies
Anonymous
Not applicable
‎2017-10-24 03:34 AM
Author

Hello,

On which build version you got this issue?

Could you please take a look at this jira issue:https://jira.talendforge.org/browse/TBD-5521 to see if you have the same situation on your side?

Best regards

Sabrina

188 Views
0 Likes
Anonymous
Not applicable
‎2017-10-24 11:36 PM
Author

Hi Sabrina,

 

I ran into this issue on Talend Data Fabric 6.4.1.

Yes, the problem seems similar as my job fails to build and generate code too.

It says that the solution is to use Talend 6.4.2 or 6.5, whereas on the Download page only Talend 6.5 was available for download as a Milestone release type. Is the current solution only applicable for Milestone release type for now?

188 Views
0 Likes
Anonymous
Not applicable
‎2017-10-25 05:11 AM
Author

Hello,

@veee wrote:

Hi Sabrina,

 

I ran into this issue on Talend Data Fabric 6.4.1.

Yes, the problem seems similar as my job fails to build and generate code too.

It says that the solution is to use Talend 6.4.2 or 6.5, whereas on the Download page only Talend 6.5 was available for download as a Milestone release type. Is the current solution only applicable for Milestone release type for now?

With your subscription solution Talend Data Fabric 6.4.1, could you please create a case on talend support portal? In this way, our colleagues from support team will schedule a webex and check your issue to see if it is possible to give a patch to you. https://login.talend.com/support-login.php.

Many thanks.

Best regards

Sabrina

188 Views
0 Likes