Not able to schedule big-data jobs(e.g.PIG,HIVE,M/R) using OOZIE

[Anonymous]

Hi,
I am very new to Talend. Is there a way to schedule Hadoop Jobs (e.g. HIVE, PIG etc.) using OOZIE plugin.
Can you please show me a example.
Regards,
Shouvanik

[Anonymous]

Hi,

@rdubois
I am using version 5.2.

But, the answer to your below statement "In 5.2, Talend only supports HortonWorks Data Platform with Oozie. In 5.3, we support much more distributions." is incorrect, because I have used Oozie with Apache distribution for Hadoop also.
I am even evaluating 5.3. Can you show me how to do it well with 5.2 also.

@esabot
I have this thread since long. If you are not able to find out, what problem I am facing, please let me know. I will again send you the query.
Regards,
Shouvanik

[Anonymous]

@rdubois
I am using version 5.2.

I mean the Hadoop distribution. Do you use HortonWorks, Cloudera, MapR ? And which version of this distribution

I am even evaluating 5.3. Can you show me how to do it well with 5.2 also.

It's not possible using the 5.2.2. You have to upgrade in 5.3.
Cheers,

[Anonymous]

Thanks for your reply.
I am now using version 5.3. And I always get the annoying error.
Deploying job to Hadoop...
Deployment failed!
The local file can not upload to Hadoop HDFS!
java.lang.reflect.InvocationTargetException

Can you please show me the way?
Regards,
Shouvanik

[Anonymous]

Hi Shouvanik

I mean the Hadoop distribution. Do you use HortonWorks, Cloudera, MapR ? And which version of this distribution

Have you answered Remy regarding your distribution of hadoop and the version of it?
Cheers,
Elisa

[Anonymous]

Hi,
I agree the Oozie logs are not very explicit. You will find additional logs browsing the oozie console and then the jobtracker logs.
Oozie console: http://hostname:11000/oozie
-> Then click on your job which failed.
-> Then browse the logs of this job.
You will be able to find details here.
The error you get gives a clue even though. The local file can't be sent to HDFS. I think about a permission issue.

[Anonymous]

Hi,
I am using version of hadoop - cdh4 and Talend Open Studio for Big data - ver 5.2.1
Regards,
Shouvanik

[Anonymous]

Hi,
The earlier problem was access issue. The expected user was "hdfs". Job is getting deployed. Hurray!!!
Deploying job to Hadoop...
Deployment complete!
Error submitting workflow job to Oozie.

The error(new) is below.
Please check if the "Job tracker end point" and "Oozie end point" are valid!
E0901. E0901: Namenode not allowed, not in Oozies whitelist
How to resolve it?
Please help.
Regards,
Shouvanik

[Anonymous]

I have checked in oozie-site.xml for the value -> its showing as
<PROPERTY>
<NAME>oozie.service.HadoopAccessorService.nameNode.whitelist</NAME>
<VALUE> </VALUE>
<DESCRIPTION>
Whitelisted job tracker for Oozie service.
</DESCRIPTION>
</PROPERTY>
No value inside. what to do.
Please help.
Regards,
Shouvanik

[Anonymous]

Hi
Have you checked out the doucmentation (https://help.talend.com/display/TalendOpenStudioforBigDataGettingStartedGuide53EN/2.1.1+How+to+set+H...) to make sure your job tracker end point and oozie end point are properly configured?
The screenshot in the documentation link shows some syntax examples.
Elisa

[Anonymous]

Yes. I have checked it. I am still without any clue. can you please help me?