Execute Spark batch job as root.user

[Anonymous]

Hi, I have a problem running a Spark batch job on MapR.

I'm currently working on a Spark batch job on a cluster with multiple nodes.

But when I ran the job, the job fails, with this message as an output:

"Yarn application has already ended! It might have been killed or unable to launch application master."

I checked the resource manager log and saw that the job failed with this message:

"Requested user root is not whitelisted and has id 0,which is below the minimum allowed 500."

I think the job fails when it runs as user "root". So is there any way I can run the job as a user besides root?

[Anonymous]

Hello,

We have a jira issue:https://jira.talendforge.org/browse/TBD-3955 about this error: "Requested user root is not whitelisted and has id 0,which is below the minimum allowed 500".

Let us know if the solution works for you.

Best regards

Sabrina

[Anonymous]

Hi Sabrina,

Thanks for responding to the problem.

If I understand correctly, the solution to the Jira issue was to use a JobServer.

But I can't find a JobServer directory in my Talend installation directory. I've only managed to find several jars which relate to JobServer.

From this link (https://help.talend.com/reader/ozfCR4oo4noN8gSKT7qP_w/TgjvBs4NXECwoSePsMB0HQ), it seems that there should be a folder named JobServer.

Do I have to do a separate installation in order to use it? Or is the JobServer directory stored somewhere else?

[Anonymous]

Hello,

Are you using talend data platform product? Free trial version? Is there any archive file name for example reads: Talend-JobServer-YYYYMMDD_HHmm-VA.B.C.zip?

Best regards

Sabrina

[Anonymous]

Hi,

I'm using a licensed version of Talend Data Fabric.

But no, I can't find a zip file under that name.

[Anonymous]

Hi,

With you licensed version of Talend Data Fabric, could you please create a case on talend support portal so that we can give you a remote assistance(webex) to see if there is any problem for your job server installation through support cycle with priority?

https://login.talend.com/support-login.php.

Best regards

Sabrina

[Anonymous]

Hi,

Okay, will try to contact the support team. Thank you Sabrina.