Hi again,
I followed the steps :
"specify the Hadoop home directory that contains the winutils.exe program
- If you don't have a local Hadoop install on Windows you can download winutils.exeand then:
- create a Hadoop home directory
- place winutils.exe in a bin directory under that Hadoop home directory
- use a system property -Dhadoop.home.dir to point to the Hadoop home directory when you start the Java process. An example:
java -D"hadoop.home.dir=C:\Users\<username>\Hadoop" -jar my.jar"
I dowloaded hadoop-common-2-2-0.bin-master containing winutils.exe, and placed it in a new directory "C:\hadoop_home\hadoop-common-2.2.0-bin-master\bin". I then set the VM argument -D"hadoop.home.dir=C:\hadoop_home\hadoop-common-2.2.0-bin-master" in Advanced parameter->JVM setting of the job (see attachment talendhadooppig2). It seems to work since I don't have the java exception "java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries." anymore. Nevertheless I still got some warnings :
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[WARN ]: org.apache.pig.PigServer - Empty string specified for jar path
My job should normally use Pig to import 2 tables from HDFS (1 main and 1 ref), do the lookup mapping, and export 2 tables to HDFS (results and rejects) in a new directory. It is running without ending, while the data flow ends on the design panel with only 1 row processed, and the output directory is not created (even erased if I create one...).
The warning are quite explicite but I don't know what to do with it... builtin-java classes are not always applicable ? how can I set the jar path for the PigServer ?
Thank you in advance,
Cordially, Sélim
talendhadooppig2.PNG
