topic Re: Problem with tHiveInput in Talend Studio

Problem with tHiveInput

mrjeffmoreira — Sat, 16 Nov 2024 11:25:54 GMT

Hello,
I would take a doubt, which I do not know what it can be.
When I connect a tHiveInput component to an output component, in my case tLogRow, he brings no data and not even starts running.
Connection with the Cluster and Hive verified, component metadata also...
What could it be?
Thank you.

Re: Problem with tHiveInput

Anonymous — Wed, 12 Nov 2014 03:08:40 GMT

Hi,
Do you mean your MapReduce job fail to execute without error message? What's your BigData product build version? Could you please share your tHiveInput component setting screenshot with us? How did you verify your Cluster and Hive?
Best regards
Sabrina

Re: Problem with tHiveInput

mrjeffmoreira — Wed, 12 Nov 2014 18:27:31 GMT

So, the thing is when I click Run i have no errors, but the main row stays "Starting" forever.
This problem does not occurs when I send a generic select like:
SELECT * FROM USERS;
But when I do something like this, specifying columns I have no answer, like this:
SELECT NAME, LASTNAME FROM USERS;
We are using Talend Big Data 5.5.1.

Re: Problem with tHiveInput

Anonymous — Tue, 18 Nov 2014 09:29:24 GMT

Hello,
SELECT * FROM USERS won't start a M/R job on the cluster side. So the data can be retrieved very quickly if you don't have a lot of data.
As soon as you define the columns you want to read in the query, a M/R job is started on the cluster side. This job could be longer. If it hangs forever, then you could find more information on the resource manager side browsing 192.168.25.86:8088.
HTH,
Rémy.

Re: Problem with tHiveInput

mrjeffmoreira — Thu, 20 Nov 2014 12:24:20 GMT

Hello,
SELECT * FROM USERS won't start a M/R job on the cluster side. So the data can be retrieved very quickly if you don't have a lot of data.
As soon as you define the columns you want to read in the query, a M/R job is started on the cluster side. This job could be longer. If it hangs forever, then you could find more information on the resource manager side browsing 192.168.25.86:8088.
HTH,
Rémy.

Hello rdubois,
When I'm running in the HortonWorks/Talend Sandbox, it works well.
But, when I try to use another machine with remote connection, just to perform that function... when I'm running remotely, it doesn't work.
I didn't say it before, but this remote machine that I'm trying to run the job ISN'T in the Hadoop Cluster, I'm just connecting remotely and trying to run the job.
In this case, will the job work well if I configure everything correctly?
Best,
Jefferson Moreira

Re: Problem with tHiveInput

Anonymous — Fri, 21 Nov 2014 10:11:30 GMT

Hello Jefferson Moreira,
Yes, there is abolutely no problem for a job to be executed remotely, from a machine which is not part of the cluster.
What error do you have when you execute from this machine?
Regards,
Rémy.

Re: Problem with tHiveInput

mrjeffmoreira — Fri, 21 Nov 2014 11:18:03 GMT

Hello Jefferson Moreira,
Yes, there is abolutely no problem for a job to be executed remotely, from a machine which is not part of the cluster.
What error do you have when you execute from this machine?
Regards,
Rémy.

Hello Rémy,
So... this is the problem, we don't have error message, the job just doesn't finish.
I don't know if you saw the screens that i sent before, is that configuration right ?
Best,
Jefferson.

Re: Problem with tHiveInput

Anonymous — Fri, 21 Nov 2014 12:13:31 GMT

Hello Jefferson,
The port you used for the resource manager looks wrong. Can you please try to replace 50300 by 8050.
HTH,
Rémy.