topic Re: Adding SerDe Jar in Hive with tHiveInput in Talend Studio

Adding SerDe Jar in Hive with tHiveInput

Anonymous — Sat, 16 Nov 2024 12:01:45 GMT

We have a table in Hive that requires a SerDe to parse the data. In other to run the query to extract the data in Talend, we need to add the SerDe Jar files prior to the SQL statement.
Using TOS Big Data v5.3, with tHiveInput, I was not able to add both 'ADD JAR /usr/lib/hive/lib/contrib.jar;' and the 'select' statement because it won't allow multiple SQL statements within the query.
The current work around is to create another physical table in Hive with create-table-as outside of Talend.
Are there other ways?

Re: Adding SerDe Jar in Hive with tHiveInput

Anonymous — Tue, 14 May 2013 17:44:02 GMT

Hi
would tLibraryLoad work?
Regards,
Brandon

Re: Adding SerDe Jar in Hive with tHiveInput

Anonymous — Mon, 20 May 2013 19:11:34 GMT

Thanks Brandon for responding. I tried the tLibraryLoad, but it seemed like the SerDe Jar needed to be added within the Hive session.
We are also playing around the idea of changing the hive-site.xml to include the contrib.jar by default.

Re: Adding SerDe Jar in Hive with tHiveInput

Anonymous — Wed, 22 May 2013 16:50:32 GMT

Hello,
I think you need to do additional things in order to add the JAR in the Hive classpath.
ADD JAR ?s3n://matthewsbucket/superudf.jar?;
CREATE TEMPORARY FUNCTION super as ?com.matthewrathbone.SuperFunction?;
You could try to use that with two tHiveRow components.
Regards,
Rémy.