How to pass tJavaRow output (i.e.sql queries) as a batch to tHiveInput or tHiveRow

[Anonymous]

Hi, 

I have a job design as below. 

tFileInputDelimited -> tFlowtoIterate -> tJavaFlex ->(Iterate) tFileInputDelimited -> tJavaRow 

I am preparing the SQL queries dynamically from a file using the above flow and the till here the flow is working fine. Now, I want to pass the output from tJavaRow as batch(i.e. all the list of queries) to tHiveInput or tHiveRow. 

I tried.. 

..-> tJavaRow -> tForEach -> (OnComponentOk) -> tHiveConnection -> tHiveRow. 

The above flow is not working for me. I am sure, I am doing something wrong here. Please correct or suggest how to proceed further. 

[fdenis]

link for each with iterate not with OnComponentOh

[Anonymous]

hi @fdenis, 

I tried, but its not working. Below is my job_design snapshot. 

Also, as i need to send all the queries at once to a Hive session, will tHiveRow work ?  As i read, tHiveRow accepts one query at a time. I may be wrong. Please correct me.

[fdenis]

what exactly is your requirement?
there is a lot of trouble on your graph!

[Anonymous]

@fdenis,

As i already explained, I have list of queries generated as an output from tJavaRow component. 

i.e. eg:-

select * from emp where emp_id = "121"

select * from dept where dept_name "XYZ"

..

and so on..

I need send those queries as batch (i.e all queries at once to the Hive session) and retrieve the result. 

[fdenis]

ok so remove tforeach.
remove OnSubjobOk link from connection
add tPreJob link with onSubjobOk to connection
and a row link(orange) from tJavaRow to thiveInput
propagate output data of tJavaRow to OutputLink
and try

[Anonymous]

I did the changes, as above. 

• From tPrejob, I only have "OnComponentOk" link available. 
• From tJavaRow, i only have "OnComponentOk" link available to connect to tHiveInput.
• Under the Query field in tHiveInput i have given the (i.e.variable from tJavaRow) variable. "outputRow.final_query" (i.e. this is the variable which holds the list of queries). 

I am getting an error - "output_row cannot be resolved to a variable". 

[fdenis]

when you right click on tJavaRow you must have row > main.
on tJavarow click on edit schema ... show me
what is your code into tjavarow?

[Anonymous]

I have tJavaRow (row->main) .. but i am not able to connect to tHiveInput as main

[fdenis]

use tflow to iterate
tJavaRow --> row --> tFlowToIterate --> iterate --> THiveInput