Talend Big Data: java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries

Problem

While you start a job in Talend Big Data, you could see the warning:

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries 

or

java.io.FileNotFoundException: The hadoop home directory (hadoop.home.dir) doesn't contain the required winutils.exe binary

 

Solution #1 (recommended)

  1. Download the file wintuils.exe eg from this page on GitHub (https://github.com/steveloughran/winutils) in the corresponding version of your Hadoop environment.
  2. Create a directory: C:\hadoop\bin
  3. Copy the previously downloaded winutlis.exe file to the C:\hadoop\bin folder
  4. Create a new environment variable: HADOOP_HOME and enter the path to the folder where the winutils.exe file is located:
    HADOOP_HOME=C:\hadoop\bin

     

  5. Uruchom ponownie Talend Studio

Solution #2

  1. Download the file wintuils.exe eg from this page on GitHub (https://github.com/steveloughran/winutils) in the corresponding version of your Hadoop environment.
  2. Create a directory: C:\hadoop\bin
  3. Copy the previously downloaded winutlis.exe file to the C:\hadoop\bin folder
  4. In the job configuration at Talend Studio, open the “Run -> Advanced” tab.
  5. In the JVM Settings section, click the new button to add a new argument.
  6. Add an argument like this: -Dhadoop.home.dir=C:\hadoop\bin
    -Dhadoop.home.dir=C:\hadoop\bin

     

Summary

Why Solution #1 is recommended? And because you only do this once and it will affect all the jobs you create.

In Solution #2, for each newly created work, add the –Dhadoop.home.dir=C:\hadoop\bin argument in the “Run -> Advanced” section.

 

If you enjoyed this post please leave the comment below or share this post on your Facebook, Twitter, LinkedIn or another social media webpage.
Thanks in advanced!

Please follow and like us:

Leave a Reply

Close Menu
Social media & sharing icons powered by UltimatelySocial

Enjoy this blog? Please spread the word :)