Talend Big Data: java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries

Problem

While you start a job in Talend Big Data, you could see the warning:

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries 

or

java.io.FileNotFoundException: The hadoop home directory (hadoop.home.dir) doesn't contain the required winutils.exe binary

 

Solution #1 (recommended)

  1. Download the file wintuils.exe eg from this page on GitHub (https://github.com/steveloughran/winutils) in the corresponding version of your Hadoop environment.
  2. Create a directory: C:\hadoop\bin
  3. Copy the previously downloaded winutlis.exe file to the C:\hadoop\bin folder
  4. Create a new environment variable: HADOOP_HOME and enter the path to the folder where the winutils.exe file is located:
    HADOOP_HOME=C:\hadoop\bin

     

  5. Uruchom ponownie Talend Studio

Solution #2

  1. Download the file wintuils.exe eg from this page on GitHub (https://github.com/steveloughran/winutils) in the corresponding version of your Hadoop environment.
  2. Create a directory: C:\hadoop\bin
  3. Copy the previously downloaded winutlis.exe file to the C:\hadoop\bin folder
  4. In the job configuration at Talend Studio, open the “Run -> Advanced” tab.
  5. In the JVM Settings section, click the new button to add a new argument.
  6. Add an argument like this: -Dhadoop.home.dir=C:\hadoop\bin
    -Dhadoop.home.dir=C:\hadoop\bin

     

Summary

Why Solution #1 is recommended? And because you only do this once and it will affect all the jobs you create.

In Solution #2, for each newly created work, add the –Dhadoop.home.dir=C:\hadoop\bin argument in the “Run -> Advanced” section.

 

If you enjoyed this post please add the comment below or share this post on your Facebook, Twitter, LinkedIn or another social media webpage.
Thanks in advanced!

Leave a Reply

avatar
  Subscribe  
Notify of
Close Menu