I have a job that is running well in local mode. But when I try to launch the same job on AWS EMR, I get the following error: This is the error as seen from the Spark log files.
Diagnostics Info: AM Container for appattempt_1525124598990_0007_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: File file:/Users/Desktop/TALEND/Talend-Studio-20180116_1512-V6.5.1/workspace/.Java/lib/metrics-core-3.1.2.jar does not exist java.io.FileNotFoundException: File file:/Users/Desktop/TALEND/Talend-Studio-20180116_1512-V6.5.1/workspace/.Java/lib/metrics-core-3.1.2.jar does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:640) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:866) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:630) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:452) at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:253) at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:361) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:359) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) For more detailed output, check the application tracking page: http://ip-10-0-0-12.ec2.internal:8088/cluster/app/application_1525124598990_0007 Then click on links to logs of each attempt.
Note: Looks like the talend job is trying to find the file "metrics-core-3.1.2.jar" from my local(developer) machine, but it is unable to find it. However I verified that the file exists and is accessible.
The metrics-core-3.1.2.jar file should be shipped within Talend Studio. Could you please create a case on talend support portal so that we can give you a remote assistance(webex) through support portal with priority?
