Configure an HDFS for Development/Testing

I am using the Go implementation of WebHDFS interface: https://github.com/vladimirvivien/gowfs. In order to test it, I need to set up an HDFS on my development computer (Mac OS X 10.8, Hadoop-2.2.0). The author Vladimir Vivien reminded two properties to enable WebHDFS:

  1. Enable dfs.webhdfs.enabled property in hdfs-site.xml
  2. Ensure hadoop.http.staticuser.user property is set in your core-site.xml.

However, those are not enough. If you see error messages like the following reported by the append operation:

Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try.

You need to add the following properties in hdfs-site.xml

  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>

  <property>
    <name>dfs.client.block.write.replace-datanode-on-failure.enable</name>
    <value>false</value>
  </property>