Get java.net.UnknownHostException when use hadoop-ha?

Question

I got an exception when i execute the command sudo -u hdfs hdfs balancer -threshold 5.

Here is the Exception.

RuntimeException: java.lang.IllegalArgumentException: java.net.UnknownHostException: nameservice1

Here is my core-site.xml.

    <property>
            <name>fs.defaultFS</name>
            <value>hdfs://nameservice1</value>
    </property>

Here is my hdfs-site.xml.

    <property>
            <name>dfs.nameservices</name>
            <value>nameservice1</value>
    </property>
    <property>
            <name>dfs.ha.namenodes.nameservice1</name>
            <value>nn1,nn2</value>
    </property>

Someone help me?

Cutinize

Answer 1

I ran into this problem when setting up HA. The problem is that I set dfs.client.failover.proxy.provider.mycluster based on the reference documentation. When I replaced mycluster with my nameservice name, everything worked!

Reference: https://issues.apache.org/jira/browse/HDFS-12109

Confine

nice, i add this property in hdfs-site.xml, and run hdfs dfs -ls /, everything is ok. – Seduction

Answer 2

You can try after putting the port number at core-site.xml.

<property>
   <name>fs.defaultFS</name>
   <value>hdfs://nameservice1:9000</value>   
</property>

And make sure your machine's /etc/hosts file has entry for nameservice1. For Example (let you machine IP is 192.168.30.102)

127.0.0.1           localhost       
192.168.30.102      nameservice1

Mohenjodaro

When configuring HA, HDFS shouldn't even attempt to resolve the name nameservice1, it should rely on dfs.namenode.rpc-address.nameservice1.nn1 (resp .nn2). The way you set it up, how do you configure two namenodes and automatic failover? – Jonquil

Answer 3

<property>
  <name>dfs.client.failover.proxy.provider.nameservice1</name>  
  <value>
     org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
  </value>
 </property>

Gaynor