Talend job not submitting to the defined YARN queue

[jatin49gupta]

Hi Experts,

We wanted to prioritize some of the talend jobs. hence we setup a dedicated YARN queue 'root.talend.high' and assigned resources to it.

We have also set the paramter as queueName: 'root.talend.high' in talend job.

However, when submitting the jobs it is always landing on 'root.default' queue as show in below logs.

Am I missing something here ? need your inputs in fixing this. Thanks in advance 

[INFO ]: org.apache.spark.deploy.yarn.Client - Application report for application_1558146131347_298872 (state: ACCEPTED)
[INFO ]: org.apache.spark.deploy.yarn.Client -
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.default
start time: 1561621900134
final status: UNDEFINED

[Anonymous]

Hello,

There are no Error and WARN from your log. Are your job able to stop automatically? Could you please upload your job setting screenshot into community which will be helpful for us to address your issue.

Please mask your sensitive data.

Best regards

Sabrina

[jatin49gupta]

Hi xdshi,

Thanks for your response.

There is no such error or warning here,job is completed successfully.

Only thing is,it is landing in "root.default" YARN queue although I have set the "queueName" to "root.talend.high".

Let me modify my question here as I have got some of my answers:

Is it possible to change the queueName for a deployed job from TAC console .

(I already verified that it is going in right queue if I update it from Talend Studio)

Thanks in advance!

[roalro]

Hi guptj72.

Where can I set Yarn queue Name parameter in Talend Job? in Spark configuration on Run Tab or where?

Thanks.

[swatisood8]

Hi,

I am facing the same problem. 

Is it possible for you to share the details of how you have specified the yarn queue on Taland? I am trying to the same and do not know an answer yet and by default, all the jobs are being submitted to the default queue. 

Regards,

Swati Sood

[swatisood8]

Hi,

I am facing the same problem. 

Is it possible for you to share the details of how you have specified the yarn queue on Taland? I am trying to the same and do not know an answer yet and by default, all the jobs are being submitted to the default queue. 

Regards,

Swati Sood

[roalro]

Hi swatisood8:

You must put the parameter in the in Spark configuration on Run Tab. If you have this option disable, change the Spark configuration to Built in and write te parameter.

I hope that it will be useful.

Regards.