Error executing hdfs namenode -format

I am new to hadoop, hdfs.. I have do the next steps:

I have started zookeeper in the three namenodes:

*vagrant@172:~$ zkServer.sh start

I can see the status:

*vagrant@172:~$ zkServer.sh status

Result Status:

JMX enabled by default
    Using config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
    Mode: follower

with jps command only appear jps and sometimes appear quaroom too:

*vagrant@172:~$ jps
    2237 Jps

I run the next command too.

* vagrant@172:~$ hdfs zkfc -formatZK

And when I execute the next, it failed:

vagrant@172:~$ hdfs namenode -format


 15/12/02 10:54:34 WARN client.QuorumJournalManager: Waited 59880 ms (timeout=60000 ms) for a response for hasSomeData. No responses yet.
15/12/02 10:54:34 WARN namenode.NameNode: Encountered exception during format: 
java.io.IOException: Timed out waiting for response from loggers
    at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:228)
    at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:899)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:171)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:940)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1382)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
15/12/02 10:54:37 FATAL namenode.NameNode: Failed to start namenode.
java.io.IOException: Timed out waiting for response from loggers
    at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:228)
    at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:899)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:171)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:940)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1382)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
15/12/02 10:54:38 INFO util.ExitUtil: Exiting with status 1
15/12/02 10:54:39 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1
************************************************************/

Then, if I run:

*vagrant@172:~$ hdfs namenode -bootstrapStandby

I have the next error:

15/12/02 10:58:45 FATAL namenode.NameNode: Failed to start namenode.
java.io.IOException: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: :0
    at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:425)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1407)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
Caused by: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: :0
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
    at org.apache.hadoop.hdfs.DFSUtil.substituteForWildcardAddress(DFSUtil.java:1171)
    at org.apache.hadoop.hdfs.DFSUtil.getInfoServerWithDefaultHost(DFSUtil.java:1138)
    at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.parseConfAndFindOtherNN(BootstrapStandby.java:397)
    at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:103)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:420)
    ... 2 more
15/12/02 10:58:46 INFO util.ExitUtil: Exiting with status 1
15/12/02 10:58:46 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1
************************************************************/

Any help for this error would be a great help for me.

My configuration are the next:

bashrc

###JAVA CONFIGURATION###
JAVA_HOME=/usr/lib/jvm/java-8-oracle
export PATH=$PATH:$JAVA_HOME/bin

###HADOOP CONFIGURATION###
HADOOP_PREFIX=/opt/hadoop-2.6.2/
export PATH=$PATH:$HADOOP_PREFIX/bin:$HADOOP_PREFIX/sbin

###ZOOKEPER###
export PATH=$PATH:/opt/zookeeper-3.4.6/bin

hdfs-site.xml

<configuration>
 <property>
  <name>dfs.replication</name>
  <value>2</value>
 </property>
 <property>
  <name>dfs.name.dir</name>
  <value>file:///hdfs/name</value>
 </property>
 <property>
  <name>dfs.data.dir</name>
  <value>file:///hdfs/data</value>
 </property>
 <property>
  <name>dfs.permissions</name>
  <value>false</value>
 </property>
 <property>
  <name>dfs.nameservices</name>
  <value>auto-ha</value>
 </property>
 <property>
  <name>dfs.ha.namenodes.auto-ha</name>
  <value>nn01,nn02</value>
 </property>
 <property>
  <name>dfs.namenode.rpc-address.auto-ha.nn01</name>
  <value>172.16.8.191:8020</value>
 </property>
 <property>
  <name>dfs.namenode.http-address.auto-ha.nn01</name>
  <value>172.16.8.191:50070</value>
 </property>
 <property>
  <name>dfs.namenode.rpc-address.auto-ha.nn02</name>
  <value>172.16.8.192:8020</value>
 </property>
 <property>
  <name>dfs.namenode.http-address.auto-ha.nn02</name>
  <value>172.16.8.192:50070</value>
 </property>
 <property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://172.16.8.191:8485;172.16.8.192:8485;172.16.8.193:8485/auto-ha</value>
 </property>
 <property>
  <name>dfs.journalnode.edits.dir</name>
  <value>/hdfs/journalnode</value>
 </property>
 <property>
  <name>dfs.ha.fencing.methods</name>
  <value>sshfence</value>
 </property>
 <property>
  <name>dfs.ha.fencing.ssh.private-key-files</name>
  <value>/home/vagrant/.ssh/id_rsa</value>
 </property>
 <property>
  <name>dfs.ha.automatic-failover.enabled.auto-ha</name>
  <value>true</value>
 </property>
 <property>
   <name>ha.zookeeper.quorum</name>
   <value>172.16.8.191:2181,172.16.8.192:2181,172.16.8.193:2181</value>
 </property>
</configuration>

core-site.xml

 <configuration>
<property>
  <name>fs.default.name</name>
  <value>hdfs://auto-ha</value>
</property>
</configuration>

zoo.cfg

 tickTime=2000

        dataDir=/opt/ZooData

        clientPort=2181

        initLimit=5
        syncLimit=2
        server.1=172.16.8.191:2888:3888
        server.2=172.16.8.192:2888:3888
        server.3=172.16.8.193:2888:3888

Answers


In file hdfs-site.xml:

*I have change all the IPs for the name of the machine. Example: 172.16.8.191 --> machine_Name1

Then in file etc/hosts:

*I have add all the IPs with their respective names

And now it is working perfectly.


Need Your Help

How can I check if an array contains a specific value in php?

php arrays

I have a PHP variable of type Array and I would like find out if it contains a specific value and let the user know that it is there. This is my array:

Ethernet CRC32 calculation - software vs algorithmic result

python algorithm ethernet crc32

I'm trying to calculate the Frame Check Sequence (FCS) of an Ethernet packet byte by byte. The polynomial is 0x104C11DB7.