Passing Parameters to Oozie Job Through tRestClient
Talend Community,
I am currently working on a project where we want to execute many oozie jobs from a talend job. We are able to deploy an oozie job that runs on a static context and it performs an action. However, what if I want to start an oozie job the the tRestClient by calling :
http://hdfs-name-node/oozie/v1/jobs?action=start
and posting to the above uri the following xml:
<configuration>
<property>
<name>
oozie.wf.application.path
</name>
<value>
{path to deployed oozie job}
</value>
</property>
...
</configuration>
I know this works. I can start the oozie job through the tRestClient by posting the xml to the aforementioned uri.
What I don't know how to do is access additional variables in the posted configuration xml from the started job. Typically, these type of things can be done by writing custom coordinator/workflow oozie scripts, but I was looking for a way that is handled directly through talend methodologies and not require custom work.
Let's say I want to access a variable such as a String called, mediaId. And I will use that mediaId to inform the job where to look in order to complete it's task. Now, I can use the configuration xml to pass a property with name mediaId and a String value of let's say "1234". When I post this configuration and look at the oozie scheduler on cloudera I find that the oozie job was started and I can actually see the variable in the details. However, I just need to be able to access that from within the job.
Thoughts? Has anyone done this before?
Cheers,
James Gilchrist
I just verified that if I navigate to the workflow.xml in the directory where I deployed my oozie job and add a couple arguments:
...<arg>--context_param</arg><arg>contextVariable=${input}</arg>...
where the configuration file posted through tRestClient contains the property
<property><name>input</name><value>input string</value></property>
then it will in fact pass in the variable dynamically and overwrite the default context parameter with the input variable.
However, this is not smooth. This would mean that a job would have to be deployed (would most likely fail or run incorrectly the first time on initial deployment) and then I would have to go manual change the workflow.xml. I don't want to do this. I would like it to build the workflow.xml for me in the right way.
Thoughts?
Hi James, I am trying to figure out how to execute Oozie work flow through Talend and found your post. I couldn't figure how to run the Oozie work flow in Talend. Could you please explain in detail how you were able to run the Oozie workflow in Talend. Thanks in advance, Sai
