Running Hadoop on Windows error 'java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset'

[TalendSolutionExpert]

Problem description

While running Hadoop from a Windows host, you get the following error message:

[ERROR] 17:20:52 org.talend.components.adlsgen2.output.AdlsGen2Output- [release] java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://wiki.apache.org/hadoop/WindowsProblems

 

Root cause

On Windows, Hadoop requires native code extensions so that it can integrate with the OS correctly for things like file access semantics and permissions.

 

For more information, see Problems running Hadoop on Windows on the Apache wiki page.

 

Solution

To resolve this issue perform one of the following options:

 

• Create a new environment variable: HADOOP_HOME and enter the path to the folder, for example:

  HADOOP_HOME=C:\hadoop\bin

  
   

• In Studio Job configuration, open the Run > Advanced settings tab. In the JVM Setting section, select the Use specific JVM arguments check box, click the New button, and add a new argument like this:

  Dhadoop.home.dir=C:\hadoop\bin