topic Re: Talend job not submitting to the defined YARN queue in Talend Studio

Talend job not submitting to the defined YARN queue

jatin49gupta — Sat, 16 Nov 2024 05:27:34 GMT

Hi Experts,

We wanted to prioritize some of the talend jobs. hence we setup a dedicated YARN queue 'root.talend.high' and assigned resources to it.

We have also set the paramter as queueName: 'root.talend.high' in talend job.

However, when submitting the jobs it is always landing on 'root.default' queue as show in below logs.

Am I missing something here ? need your inputs in fixing this. Thanks in advance

[INFO ]: org.apache.spark.deploy.yarn.Client - Application report for application_1558146131347_298872 (state: ACCEPTED)
[INFO ]: org.apache.spark.deploy.yarn.Client -
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.default
start time: 1561621900134
final status: UNDEFINED

Re: Talend job not submitting to the defined YARN queue

Anonymous — Tue, 02 Jul 2019 09:52:04 GMT

Hello,

There are no Error and WARN from your log. Are your job able to stop automatically? Could you please upload your job setting screenshot into community which will be helpful for us to address your issue.

Please mask your sensitive data.

Best regards

Sabrina

Re: Talend job not submitting to the defined YARN queue

jatin49gupta — Tue, 02 Jul 2019 10:04:56 GMT

Hi xdshi,

Thanks for your response.

There is no such error or warning here,job is completed successfully.

Only thing is,it is landing in "root.default" YARN queue although I have set the "queueName" to "root.talend.high".

Let me modify my question here as I have got some of my answers:

Is it possible to change the queueName for a deployed job from TAC console .

(I already verified that it is going in right queue if I update it from Talend Studio)

Thanks in advance!

Re: Talend job not submitting to the defined YARN queue

roalro — Wed, 20 May 2020 10:29:24 GMT

Hi guptj72.

Where can I set Yarn queue Name parameter in Talend Job? in Spark configuration on Run Tab or where?

Thanks.

Re: Talend job not submitting to the defined YARN queue

swatisood8 — Wed, 08 Jul 2020 13:29:36 GMT

Hi,

I am facing the same problem.

Is it possible for you to share the details of how you have specified the yarn queue on Taland? I am trying to the same and do not know an answer yet and by default, all the jobs are being submitted to the default queue.

Regards,

Swati Sood

Re: Talend job not submitting to the defined YARN queue

swatisood8 — Wed, 08 Jul 2020 13:29:36 GMT

Hi,

I am facing the same problem.

Regards,

Swati Sood

Re: Talend job not submitting to the defined YARN queue

roalro — Thu, 23 Jul 2020 10:27:34 GMT

Hi swatisood8:

You must put the parameter in the in Spark configuration on Run Tab. If you have this option disable, change the Spark configuration to Built in and write te parameter.

I hope that it will be useful.

Regards.