Not able to schedule big-data jobs(e.g.PIG,HIVE,M/R) using OOZIE

[Anonymous]

Hi,
I am very new to Talend. Is there a way to schedule Hadoop Jobs (e.g. HIVE, PIG etc.) using OOZIE plugin.
Can you please show me a example.
Regards,
Shouvanik

[Anonymous]

Hi,

I also tried the below.
Copied all hadoop jars from server location inside -> C:\Users\shouvanik.haldar\Downloads\TOS_BD-r95165-V5.2.1\TOS_BD-r95165-V5.2.1\plugins\org.talend.designer.components.bigdata_5.2.1.r95165\components\tHBaseConnection folder in addition to all the jars residing there.

Did you choose Cloudera distribution and HBase version Cloudera CDH 4 before the modification using the original component? To be honest, I have not a good idea on bigdata_5.2.1.r95165.
Best regards
Sabrina

[Anonymous]

Hi Sabrina,
I did as you had shown in the screenshot. Can you please redirect to some one who can help me with 5.2.1 version of Talend
Regards,
Shouvanik

[Anonymous]

Hi Sabrina,
Can you please help?
Regards,
Shouvanik

[Anonymous]

Hi Sabrina,
I am using Talend 5.3.0 now as advised by you. But while I try to schedule it in OOZIE, I get the following error
Deploying job to Hadoop...
Deployment failed!
The local file can not upload to Hadoop HDFS!
java.lang.reflect.InvocationTargetException

And when I use "Custom-unsopported". I get the following error
Import custom definition failed
java.lang.IllegalArgumentException: InputStream cannot be null
javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
org.talend.core.hadoop.version.custom.HadoopCustomLibrariesUtil.readZipFile(HadoopCustomLibrariesUtil.java:312)
org.talend.core.hadoop.version.custom.HadoopVersionDialog.getImportLibLibraries(HadoopVersionDialog.java:426)
org.talend.core.hadoop.version.custom.HadoopCustomVersionDefineDialog$12$1.run(HadoopCustomVersionDefineDialog.java:547)
org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
org.eclipse.jface.operation.ModalContext$ModalContextThread.block(ModalContext.java:173)
org.eclipse.jface.operation.ModalContext.run(ModalContext.java:388)
org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:507)
org.talend.core.hadoop.version.custom.HadoopCustomVersionDefineDialog.doImportLibs(HadoopCustomVersionDefineDialog.java:575)
org.talend.core.hadoop.version.custom.HadoopCustomVersionDefineDialog.access$4(HadoopCustomVersionDefineDialog.java:509)
org.talend.core.hadoop.version.custom.HadoopCustomVersionDefineDialog$1.run(HadoopCustomVersionDefineDialog.java:165)
org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
org.eclipse.jface.window.Window.runEventLoop(Window.java:825)
org.eclipse.jface.window.Window.open(Window.java:801)
org.talend.designer.core.ui.editor.properties.controllers.HadoopJarSetupController$1.widgetSelected(HadoopJarSetupController.java:131)
org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:234)
org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4066)
org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3657)
org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2640)
org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2604)
org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
org.talend.rcp.intro.Application.start(Application.java:133)
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:619)
org.eclipse.equinox.launcher.Main.basicRun(Main.java:574)
org.eclipse.equinox.launcher.Main.run(Main.java:1407)
org.eclipse.equinox.launcher.Main.main(Main.java:1383)


Please help.
Regards,
Shouvanik

[Anonymous]

Hi Sabrina,
I am facing error with talend open studio for big data. The following error is
Deploying job to Hadoop...
Deployment failed!
The local file can not upload to Hadoop HDFS!
java.lang.reflect.InvocationTargetException

Regards,
Shouvanik

[Anonymous]

When I check connection, it says "Connection failure. You must change the HDFS Settings.
Cannot connect to HDFS "hdfs://pofmv1145". Please check the connection parameters."
But I am able to see the hdfs file structure. What can be the reason? Please help.
Regards,
Shouvanik

[Anonymous]

Hadoop is running. That's for sure.
Sabrina,
Please reply

[Anonymous]

Hi,
I was under impression Talend is the answer to every solution. We are evaluating it, but find so many challenges to run even small things. The forum is not swift in answering questions. I am struggling with the above, but no help is coming?
Regards,
SHouvanik

[Anonymous]

Hi,
I have not read all the thread. Nevertheless, the issues you meet means the hadoop version within Talend and the hadoop version within the server mismatch.
In 5.2, Talend only supports HortonWorks Data Platform with Oozie. In 5.3, we support much more distributions.
Which distribution are you using?

[Anonymous]

Regarding the responsiveness on the Forum, it depends on the number of question which have been asked by the Community. Our team does its best to answer as promptly as possible. Sometimes we need to request support from other internal/dev teams which may not be always available on the same time zone.
If your problem is a blocker then mention it clearly in the message, so that we can filter and prioritize this.
Cheers,
Elisa