Talend Big data job is not getting submitted to Azure HD Insight cluster
Hello,
We have a job that has executed successfully in our test environment but failed in production.
The .jar file is not getting uploaded it seems. The .jar file is about 9Mb in size.
Cluster - Microsoft Azure HDinsight
Java - 1.8
Talend 6.4
21/10/21 07:13:03 INFO Client: Preparing resources for our AM container
21/10/21 07:13:06 INFO Client: Uploading resource wasb://user1@abcd.blob.core.windows.net/jar/bd_pr_ams_dstr_dec_logic_block3_0_1.jar->wasbs://user1@abcd.blob.core.windows.net/user/livy/.sparkStaging/application_163818762_10785/bd_pr_ams_dstr_dec_logic_block3_0_1.jar
21/10/21 07:13:07 WARN AzureFileSystemThreadPoolExecutor: Disabling threads forDelete operation as thread count 0 is <= 1
21/10/21 07:13:07 INFO AzureFileSystemThreadPoolExecutor: Time taken for Deleteoperation is: 2 ms with threads: 0
21/10/21 07:13:07 INFO Client: Deleted staging directory wasbs://user1@abcd.blob.core.windows.net/user/livy/.sparkStaging/application_1632721818762_10785
Exception in thread "main" java.io.FileNotFoundException: wasb://user1@abcd.blob.core.windows.net/jar/bd_pr_ams_dstr_dec_logic_block3_0_1.jar: No such file or directory.
Is Job running successfully from your local development machine? Have you tried to solve changing your path manually and moving separately/manually the missing jar file to the server to see if it works?
