Function of “hadoop namenode -format” command

Question

I am following the online tutorial to learn Hadoop and now I am installing Hadoop on my system. After configuring all the files in Hadoop. Now I am executing

hadoop namenode -format

After which I am getting the following message:

hadoop-1.2.1$ bin/hadoop namenode -format

14/11/26 12:37:16 INFO namenode.NameNode: STARTUP_MSG: 

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = localhost/127.0.0.1

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 1.2.1

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG:   java = 1.6.0_45

************************************************************/

14/11/26 12:37:17 INFO util.GSet: Computing capacity for map BlocksMap

14/11/26 12:37:17 INFO util.GSet: VM type       = 64-bit

14/11/26 12:37:17 INFO util.GSet: 2.0% max memory = 932118528

14/11/26 12:37:17 INFO util.GSet: capacity      = 2^21 = 2097152 entries

14/11/26 12:37:17 INFO util.GSet: recommended=2097152, actual=2097152

14/11/26 12:37:17 INFO namenode.FSNamesystem: fsOwner=vinay

14/11/26 12:37:17 INFO namenode.FSNamesystem: supergroup=supergroup

14/11/26 12:37:17 INFO namenode.FSNamesystem: isPermissionEnabled=true

14/11/26 12:37:17 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100

14/11/26 12:37:17 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)

14/11/26 12:37:17 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0

14/11/26 12:37:17 INFO namenode.NameNode: Caching file names occuring more than 10 times 

14/11/26 12:37:17 INFO common.Storage: Image file /tmp/hadoop-vinay/dfs/name/current/fsimage of size 115 bytes saved in 0 seconds.

14/11/26 12:37:18 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits

14/11/26 12:37:18 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits

14/11/26 12:37:18 INFO common.Storage: Storage directory /tmp/hadoop-vinay/dfs/name has been successfully formatted.

14/11/26 12:37:18 INFO namenode.NameNode: SHUTDOWN_MSG: 

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at chaitanya-OptiPlex-3010/127.0.0.8

************************************************************/

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command?

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command.

Answer 1

Basically Hadoop namenode -format does following:

• Fsimage files which contain filesystem images which are basically the metadata file in your NameNode. And edit logs are the files that contain the recent changes in the file system, which is later merged in the fsimage.  
• Formatting the namenode deletes the information from the namenode directory. The NameNode directory is specified in hdfs-site.xml file in dfs.namenode.name.dir property.
• Formatting the file system means initializing the directory specified by the dfs.name.dir variable.
• Never format, up and running Hadoop filesystem. You will lose all your data stored in the HDFS.