topic Re: Registering Hive UDF's (User Defined Functions) in Talend Studio

Registering Hive UDF's (User Defined Functions)

Anonymous — Sat, 16 Nov 2024 12:49:15 GMT

I have successfully set up and used a Hive input connector to connect to Hadoop - running in Amazon EMR (by using a JDBCInput connector and specifying all the required Hive/Hadoop libs).
The standard Hive queries work fine.
My only issue is that I cannot figure out how to register and use my UDF's as part of a job flow.
ie

add jar s3://path/libs/my_lib.jar;
create temporary function my_func as 'mylib.MyFunction';
select my_func(col_a) from ...

My UDF queries successfully run in the SQLBuilderDialog but when I attempt to run them as part of a job, the job returns almost instantly without error - with output being an empty file with the just the column headers.
Can someone please point in the correct direction for registering and using UDF's
Im using TOS 4.2.2

Re: Registering Hive UDF's (User Defined Functions)

Anonymous — Mon, 11 Jul 2011 13:34:24 GMT

Hi
I am interesting in your job, can you export the job and send it to me?
Best regards
Shong

Re: Registering Hive UDF's (User Defined Functions)

Anonymous — Fri, 02 Dec 2011 15:05:31 GMT

try putting the first two line "Add jar" and "create function" in seperate jdbc row calls and then issue the select using tjdbcinput. see if that helps.
KArthik

Re: Registering Hive UDF's (User Defined Functions)

Anonymous — Fri, 24 Jan 2014 10:52:28 GMT

any solution?
I have the same issue when I'm trying to use a function stored in HIVE
How to register the function in hive session? with thiverow first? or another way to do?
In shell: first:
create temporary function findInMsg as 'com.r9.scrub.hive.FindInMessage'"
and after: select findInMsg('local',message) from table r
I'm trying to do the same with thiverow / thiveinput but it doesn't work
thiverow:
create temporary function findInMsg as 'com.scrub.hive.FindInMessage'
> subjob ok
> tinputrow
select
findInMsg('local',message),
from
table r

Any idea?
My error:
Exception in component tHiveInput_2
java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:194)

Re: Registering Hive UDF's (User Defined Functions)

Anonymous — Wed, 12 Oct 2016 11:32:51 GMT

I know its a very very old thread but was there any solution found on this issue?