Talend Big data job is not getting submitted to Azure HD Insight cluster

TomG1 — Fri, 15 Nov 2024 23:36:29 GMT

Hello,

We have a job that has executed successfully in our test environment but failed in production.

The .jar file is not getting uploaded it seems. The .jar file is about 9Mb in size.

Cluster - Microsoft Azure HDinsight

Java - 1.8

Talend 6.4

21/10/21 07:13:03 INFO Client: Preparing resources for our AM container

21/10/21 07:13:06 INFO Client: Uploading resource wasb://user1@abcd.blob.core.windows.net/jar/bd_pr_ams_dstr_dec_logic_block3_0_1.jar->wasbs://user1@abcd.blob.core.windows.net/user/livy/.sparkStaging/application_163818762_10785/bd_pr_ams_dstr_dec_logic_block3_0_1.jar

21/10/21 07:13:07 WARN AzureFileSystemThreadPoolExecutor: Disabling threads forDelete operation as thread count 0 is <= 1

21/10/21 07:13:07 INFO AzureFileSystemThreadPoolExecutor: Time taken for Deleteoperation is: 2 ms with threads: 0

21/10/21 07:13:07 INFO Client: Deleted staging directory wasbs://user1@abcd.blob.core.windows.net/user/livy/.sparkStaging/application_1632721818762_10785

Exception in thread "main" java.io.FileNotFoundException: wasb://user1@abcd.blob.core.windows.net/jar/bd_pr_ams_dstr_dec_logic_block3_0_1.jar: No such file or directory.

Please help

Re: Talend Big data job is not getting submitted to Azure HD Insight cluster

Anonymous — Fri, 22 Oct 2021 03:22:57 GMT

Hello,

Is Job running successfully from your local development machine? Have you tried to solve changing your path manually and moving separately/manually the missing jar file to the server to see if it works?

Best regards

Sabrina

topic Talend Big data job is not getting submitted to Azure HD Insight cluster in Talend Studio

Talend Big data job is not getting submitted to Azure HD Insight cluster

Re: Talend Big data job is not getting submitted to Azure HD Insight cluster