topic Re: Exception in thread "main" java.lang.UnsatisfiedLinkError: in Talend Studio

Exception in thread "main" java.lang.UnsatisfiedLinkError:

_AnonymousUser — Sat, 16 Nov 2024 10:52:31 GMT

Hi,
We have TOS BD 6.1.1 installed on Windows 7 64 bit - while trying to write Windos local file to HDFS bu using tHDFSOutput - facing below error. Please help ASAP.
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
                at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
                at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
                at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
                at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
                at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
                at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:144)
                at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2250)
                at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
                at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
                at sun.nio.cs.StreamEncoder.implClose(Unknown Source)
                at sun.nio.cs.StreamEncoder.close(Unknown Source)
                at java.io.OutputStreamWriter.close(Unknown Source
at java.io.BufferedWriter.close(Unknown Source)
                at local_project.test_0_1.Test.tFileInputExcel_1Process(Test.java:1308)
                at local_project.test_0_1.Test.tHDFSConnection_1Process(Test.java:374)
                at local_project.test_0_1.Test.runJobInTOS(Test.java:1567)
                at local_project.test_0_1.Test.main(Test.java:1424)
Many Thanks
Regards,
Kiran G.

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Anonymous — Fri, 08 Jan 2016 03:21:43 GMT

Hi Kiran G,
Could you please take a look at this jira issue: https://jira.talendforge.org/browse/TBD-2462?
Best regards
Sabrina

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

_AnonymousUser — Fri, 08 Jan 2016 06:41:05 GMT

Hi Sabrina,
many thanks for quick reply.
We have gone through the thread unfortunately we don't see any solution
So this is because of hadoop.dll native library missing or not compatible.
We are connecting to a remote server where Hadoop 2.6.0 CDH5.4 YARN installed - it means we need Windows 7 64 bit supporting Hadoop native binary/library files and same we have to mention in HADOOP_HOME system variable.
If my above understanding is correct, could you please help us with required native library/binary files for windows 64 bit ?
Please suggest what has to be done to get rid of this issue. Many Thanks.
Regards,
Kiran G.

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

_AnonymousUser — Fri, 08 Jan 2016 13:38:05 GMT

Hi ,
I was able to download hadoop.dll and winutils.exe from github for Hadoop 2.6 Windows common. However still facing the same issue. Please suggest.
Kiran G.

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Anonymous — Mon, 11 Jan 2016 07:00:41 GMT

Hi,
Could you please try this:
1 - Download http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe and put it in a folder bin folder in your windows machine. For example: C:/tmp/winutils/bin/winutils.exe
2 - In Talend, before executing a job, go in the Advanced Parameters tab of the Run view and check the "Use specific JVM argument". Then, click on new in order to add a new entry:
-> -Dhadoop.home.dir=C:/tmp/winutils
3 - Save and execute your job.
https://jira.talendforge.org/browse/TBD-1412
Best regards
Sabrina