Execute hive script (.HQL) from talend

[Anonymous]

Hi,
Is there any way that i can just execute the hive script which is there on hdfs from talend?
All i can see in talend is component to write hive queries like thiveconncetion,thiveload etc... but none of them have way to execute hive script.
you can easily do this in oozie editor (Hue) by calling and executing hive script.
is this possible in talend?
Please advice!
Thanks

[Anonymous]

Hi nairgokul70,
Of course ,you can execute your hive script in Talend,using tHiveRow component maybe can help you .
Regards
jjzhou

[Anonymous]

Hi,
Can you please explain how?
i am not seeing anything in thiverow which can call a script?

Thanks

[willm1]

Just like for any other Talend 'row' component, whatever you can do in Hive QL directly, you can do from Talend in the query window - from selecting records to altering tables etc...

[Anonymous]

Hi,
Thanks for reply,
But i am trying to execute hive script in row component and its not working.
I have script on hdfs file system of my hadoop cluster
and i just want to execute that script. usually to execute hive script you just say
hive -f /pathtofile/script.hql
i am trying to do this row component but i am getting an error stating
hive cannot be resolved to a variable
f cannot be resolved to a variable
Syntax error on token ""/user/hive/warehouse/queries/vgt_repository/loaddata/IncrementalLoad_AccountLog.hql"", delete this token
I tried with "" and with double quotes etc but its still not working.
Please suggest.

[willm1]

Just a thought - I wonder if tSystem might work... (example calling a shell script attached)

[Anonymous]

Agree with @willm.
If you want to execute the hive command "hive -f /pathtofile/script.hql", the tSystem will be required, and a local installation of Hive will be required.

[Anonymous]

Hello,
a local installation of Hive will be required!
what you mean by above? i am executing the talend from machine different from hadoop cluster. when you say local does that mean i have to have hive installed on the machine from where i am running talend?
If that is the case the whole purpose of using talend as platform independent tool gets defeated!. Also, if i have use to bash like component i.e. tsystem component than the point of having all the hive components also gets defeated!
currently, i have divided all the scripts into different thiverow components and executing them. But now i am ending up with many components in the same job!
In my personal opinion executing a script file is very basic requirement!. I am evaluating talend for client with big data requirements and things like this is hard to explain to the client!
The java api have already class called
java.util.List<JAVA.LANG.STRING> executeScript(HiveScript script)
throws org.springframework.dao.DataAccessException
why isnt the above implemented in any of the component?
Regards
Gokul

[Anonymous]

Hello,
When you execute: "hive -f /pathtofile/script.hql", you call the hive CLI, which indeed requires hive to be installed locally. It's nevertheless not how we usually do with Talend.
We don't have a component which allows executing a Hive QL file currently.
Does your HQL file contain 1 query per line ?

[willm1]

Gokul - pardon me, because I'm sure you already thought of this - but what about deploying your Talend job on the server running Hive?
In the example I showed with the sh script, I was developing on a Windows machine, and I'd independently tested the sh script on a Linux box and it worked. Once I deployed the completed job on the Linux box, it worked as designed... Same thing if you're using TAC, you could install a Job Server on a box running Hive and constrain this job to only run on that Server.