winutils-hadoop-6.2.0.exe in Big Data/Spark jobs (Talend Data Fabric)

[Anonymous]

Hi!
I'm new to Big Data in Talend so I've created a very simple 'Big Data Batch' job using Spark framework. On attempting to run this job Talend raises a pop-up window requesting a 'jar' which actually is winutils-hadoop-6.2.0.exe, not a jar. Due to a proxy issue I don't have an option to download and install this file from Talend, so I got it from the libraries repository of talend-update site. Once installed I got a "Could not find or load main class" error when running the job. This might be due to that the file seems to be for 64 bits while I am running Talend in a 32 bit computer. I was wondering if you could provide a 32 bit version of this file as well.
Besides, it is strange to me that I have to install a jar that actually is a *.exe file. Could you please confirm that copy winutils-hadoop-6.2.0.exe to C:\Talend-Studio-20150908_
1633-V6.0.1\configuration\lib\java and then select it from there when Talend requests it is the right way to do it?
Thank you

[Anonymous]

Hi 
Can you upload a screenshot of your job design? It will be helpful for me to reproduce the problem.
Thanks!

[Anonymous]

Hi,
Find attached the screenshot. I finally reduced my initial job to just a tLogRow to isolate the problem. Here is the error:
Error: no se ha encontrado o cargado la clase principal => Error: Could not find or load main class
Thanks
 

[Anonymous]

When this occurs I get this error in .log file under workspace\.metadata
!ENTRY org.talend.platform.logging 4 0 2015-12-11 10:34:28.652
!MESSAGE 2015-12-11 10:34:28,648 ERROR org.talend.commons.exception.CommonExceptionHandler  - zip file is empty
!STACK 0
java.util.zip.ZipException: zip file is empty
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at java.util.zip.ZipFile.<init>(Unknown Source)
    at org.talend.repository.ui.utils.ZipToFile.unZipFile(ZipToFile.java:211)
    at org.talend.designer.runprocess.java.AbstractJavaProcessor.unzipProcess(AbstractJavaProcessor.java:355)
    at org.talend.designer.runprocess.java.AbstractJavaProcessor.unzipAndDeploy(AbstractJavaProcessor.java:339)
    at org.talend.designer.runprocess.java.AbstractJavaProcessor.run(AbstractJavaProcessor.java:200)
    at org.talend.designer.core.runprocess.Processor.run(Processor.java:159)
    at org.talend.designer.runprocess.RunProcessContext$1$1$1.run(RunProcessContext.java:583)
    at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
    at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:136)
    at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4147)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3764)
    at org.talend.commons.ui.swt.dialogs.EventLoopProgressMonitor.runEventLoop(EventLoopProgressMonitor.java:123)
    at org.talend.commons.ui.swt.dialogs.EventLoopProgressMonitor.isCanceled(EventLoopProgressMonitor.java:100)
    at org.talend.designer.runprocess.RunProcessContext$1.run(RunProcessContext.java:628)
    at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:466)
    at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:374)
    at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:527)
    at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:284)
    at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1190)
    at org.talend.designer.runprocess.RunProcessContext.exec(RunProcessContext.java:524)
    at org.talend.designer.runprocess.remote.RunRemoteProcessContext.exec(RunRemoteProcessContext.java:74)
    at org.talend.designer.runprocess.ui.ProcessComposite.exec(ProcessComposite.java:1450)
    at org.talend.designer.runprocess.ui.ProcessComposite.addInHistoryRunningList(ProcessComposite.java:1401)
    at org.talend.designer.runprocess.ui.ProcessComposite.execButtonPressed(ProcessComposite.java:935)
    at org.talend.designer.runprocess.ui.ProcessComposite.execRun(ProcessComposite.java:674)
    at org.talend.designer.runprocess.ui.ProcessComposite$1.widgetSelected(ProcessComposite.java:395)
    at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
    at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
    at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
    at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
    at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
    at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
    at org.talend.rcp.intro.Application.start(Application.java:183)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
!ENTRY org.talend.platform.logging 1 0 2015-12-11 10:34:28.798
!MESSAGE 2015-12-11 10:34:28,797 INFO  org.talend.designer.core.runprocess.Processor  - Command line: C:/Program Files/Java/jre1.8.0_51/bin/java.exe -Xms256M -Xmx1024M -Djava.library.path=C:\hadoop-2.7.1_32\hadoop-2.7.1\bin -cp ../lib/talend-spark-assembly-1.3.0-cdh5.4.0-hadoop2.6.0-cdh5.4.0.jar;../lib/scala-library-2.10.4.jar;../lib/slf4j-api-1.7.5.jar;../lib/commons-cli-1.2.jar;../lib/hadoop-yarn-server-web-proxy-2.6.0-cdh5.4.0.jar;../lib/dom4j-1.6.1.jar;../lib/jackson-core-asl-1.9.13.jar;../lib/antlr-runtime-3.5.2.jar;../lib/hadoop-hdfs-2.6.0-cdh5.4.0.jar;../lib/commons-configuration-1.6.jar;../lib/slf4j-log4j12-1.7.5.jar;../lib/hadoop-yarn-client-2.6.0-cdh5.4.0.jar;../lib/htrace-core-3.0.4.jar;../lib/jackson-core-2.3.2.jar;../lib/winutils-hadoop-2.6.0.exe;../lib/hadoop-yarn-api-2.6.0-cdh5.4.0.jar;../lib/hadoop-yarn-common-2.6.0-cdh5.4.0.jar;../lib/commons-codec-1.4.jar;../lib/hadoop-auth-2.6.0-cdh5.4.0.jar;../lib/commons-collections-3.2.1.jar;../lib/protobuf-java-2.5.0.jar;../lib/snappy-java-1.0.5.jar;../lib/guava-11.0.2.jar;../lib/jackson-databind-2.3.2.jar;../lib/jackson-mapper-asl-1.9.13.jar;../lib/org.talend.dataquality.parser.jar;../lib/avro-1.7.7.jar;../lib/commons-lang-2.6.jar;../lib/talend_file_enhanced_20070724.jar;../lib/avro-mapred-1.7.7.jar;../lib/hadoop-common-2.6.0-cdh5.4.0.jar;../lib/avro-compiler-1.7.7.jar;../lib/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.0.jar;../lib/log4j-1.2.16.jar;../lib/snappy-java-1.0.4.1.jar;../lib/jackson-annotations-2.3.0.jar;../lib/hadoop-mapreduce-client-common-2.6.0-cdh5.4.0.jar;../lib/hadoop-mapreduce-client-core-2.6.0-cdh5.4.0.jar;../lib/talend-mapred-lib.jar;../lib/routines.jar;../lib/camel-core-2.15.3.jar;.;prueba_spark_0_1.jar; bbva_4_12_13_10.prueba_spark_0_1.prueba_spark -libjars ../lib/scala-library-2.10.4.jar,../lib/snappy-java-1.0.5.jar,../lib/jackson-mapper-asl-1.9.13.jar,../lib/org.talend.dataquality.parser.jar,../lib/avro-1.7.7.jar,../lib/talend_file_enhanced_20070724.jar,../lib/avro-mapred-1.7.7.jar,../lib/avro-compiler-1.7.7.jar,../lib/dom4j-1.6.1.jar,../lib/log4j-1.2.16.jar,../lib/jackson-core-asl-1.9.13.jar,../lib/antlr-runtime-3.5.2.jar,../lib/talend-mapred-lib.jar,../lib/routines.jar,../lib/camel-core-2.15.3.jar --context=Default --stat_port=3579 %*