Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Submitting Spark Job using Talend tSSH component not working as requir

I have created and spark job and when I running that job directly from console its taking 5 min to completed but when I am submitting the same job using Talend tSSH component its taking around 2 Hours and not getting finished.
Please help me on the above.
Labels (2)
5 Replies
Anonymous
Not applicable
Author

Hi,
when I am submitting the same job using Talend tSSH component

Is there any error message printed on console? How did you set your tSSH component?
Best regards
Sabrina
Anonymous
Not applicable
Author

It Is not throwing any error in console and job.Attached configuration for tSSH component where we are giving Hostname,Username and password via context variables and in the commands string we are using the spark-submit command.
Anonymous
Not applicable
Author

configuration screen shoot
_AnonymousUser
Specialist III
Specialist III

Did you found a solution for that? I am facing the same issue.
Best,
Marlena
Anonymous
Not applicable
Author

Hi Sabrina,

This issue is occurring for me too. The job is getting stuck and we are able to see it stuck in "running process in linux" for hours. It is not getting killed but the control is not getting reverted to Talend when executing spark-submit from TSSH. Initial analysis suggests issue with logging streaming logs.
Thanks,
Akash Jaiswal