Login/Register
DataNode is Not Starting in singlenode hadoop 2.6.0
Asked Answered
hadoophadoop2hadoop-plugins
F

11

21

I installed hadoop 2.6.0 in my laptop running Ubuntu 14.04LTS. I successfully started the hadoop daemons by running start-all.sh and I run a WourdCount example successfully, then I tried to run a jar example that didn't work with me so I decide to format using hadoop namenode -format and start all over again but when I start all daemons using start-dfs.sh && start-yarn.sh then jps all daemons runs but not the datanode as shown bellow:

hdferas@feras-Latitude-E4310:/usr/local/hadoop$ jps
12628 NodeManager
12110 NameNode
12533 ResourceManager
13335 Jps
12376 SecondaryNameNode

How to solve that?

Flagellate answered 20/3, 2015 at 12:45 Comment(1)
Is this an actual programming question? Please Take The Tour – Kalahari
C
60

I have faced this issue and it is very easy to solve. Your datanode is not starting because after your namenode and datanode started running you formatted the namenode again. That means you have cleared the metadata from namenode. Now the files which you have stored for running the word count are still in the datanode and datanode has no idea where to send the block reports since you formatted the namenode so it will not start.

Here are the things you need to do to fix it. Stop all the Hadoop services (stop-all.sh) and close any active ssh connections.

cat /usr/local/hadoop/etc/hadoop/hdfs-site.xml

This step is important, see where datanode's data is gettting stored. It is the value associated for datanode.data.dir. For me it is /usr/local/hadoop/hadoop_data/hdfs/datanode. Open your terminal and navigate to above directory and delete the directory named current which will be there under that directory. Make sure you are only deleting the "current" directory.

sudo rm -r /usr/local/hadoop/hadoop_data/hdfs/datanode/current

Now format the namenode and check whether everything is fine.

hadoop namenode -format

say yes if it asks you for anything.

jps

Hope my answer solves the issue. If it doesn't let me know.

Little advice: Don't format your namenode. Without namenode there is no way to reconstruct the data. If your wordcount is not running that is some other problem.

Collusive answered 21/3, 2015 at 9:18 Comment(3)
Thanks, this helps. – Psychoneurotic
I still didn't get Data node running. – Vaulted
why is that "cat hdfs-site.xml" needed? I solved without cat. and I want to know the reason why this happens in the first place. For me the reason is that I previously setup hadoop using the same namenode dir and data dir as the previous running(I am just trying out installation process). Deleting the previous namenode dir and data dir works without installation issue – Organize
F
4

I had this issue when formatting namenode too. What i did to solve the issue was:

  1. Find your dfs.name.dir location. Consider for example, your dfs.name.dir is /home/hadoop/hdfs.

(a) Now go to, /home/hadoop/hdfs/current.

(b) Search for the file VERSION. Open it using a text editor.

(c) There will be a line namespaceID=122684525 (122684525 is my ID, yours will be different). Note the ID down.

  1. Now find your hadoop.tmp.dir location. Mine is /home/hadoop/temp.

(a) Go to /home/hadoop/temp/dfs/data/current.

(b) Search for the file VERSION and open it using a text editor.

(c) There will be a line namespaceID=. The namespaceID in this file and previous one must be same.

(d) This is the main reason why my datanode was not started. I made them both same and now datanode starts fine.

Note: copy the namespaceID from /home/hadoop/hdfs/current/VERSION to /home/hadoop/temp/dfs/data/current/VERSION. Dont do it in reverse.

  1. Now do start-dfs.sh && start-yarn.sh. Datanode will be started.
Feud answered 24/3, 2015 at 11:2 Comment(1)
How to locate the hadoop temp directory? What is it's default location? – Lovato
M
4

You Just need To Remove All The Contents Of DataNode Folder And Format The Datanode By Using The Following Command

hadoop namenode -format
Muskeg answered 3/6, 2016 at 8:5 Comment(0)
E
3

Even I had same issue and checked the log and found below error

Exception - Datanode log FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain java.io.IOException: All directories in dfs.datanode.data.dir are invalid: "/usr/local/hadoop_store/hdfs/datanode/

Ran the below command to resolve the issue

sudo chown -R hduser:hadoop /usr/local/hadoop_store

Note - I have create the namenode and datanode under the path /usr/local/hadoop_store

Expertize answered 6/4, 2015 at 2:12 Comment(1)
what log file? I checked "localhost: starting datanode, logging to /home/jgu/local/src/hadoop/hadoop-2.7.5/logs/hadoop-jgu-datanode-research3-pc.out" but did not see anything useful – Organize
E
2

The above problem is occurred due to format the namenode (hadoop namenode -format) without stopping the dfs and yarn daemons. While formating namenode, the question given below is appeared and you press Y key for this. Re-format filesystem in Storage Directory /tmp/hadoop-root/dfs/name ? (Y or N)

Solution, You need to delete the files within the current(directory name) directory of dfs.name.dir, you mention in hdfs.site.xml. In my system dfs.name.dir is available in /tmp/hadoop-root/dfs/name/current. rm -r /tmp/hadoop-root/dfs/name/current By using the above comment I removed files inside in the current directory. Make sure you are only deleting the "current" directory.Again format the namenode after stopped the dfs and yarn daemons (stop-dfs.sh & stop-yarn.sh). Now datanode will start normally!!

Epictetus answered 30/6, 2016 at 11:26 Comment(0)
Q
0

at core-site.xml check for absolute path of temp directory, If this is not pointed correctly or not created (mkdir). The data node cant be started.

Quezada answered 7/3, 2016 at 17:30 Comment(0)
S
0

add below property in yarn-site.xml

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
Styptic answered 25/3, 2016 at 8:48 Comment(0)
J
0

not the right way to do it. but surely works~

remove files from your datanode ,namenode and tmp folder. any files/folders created inside these are owned by hadoop and may have some reference to the last run datanode details which may have failed or locked due to which the datanode does not star at the next attempt

Jea answered 30/3, 2016 at 2:41 Comment(0)
P
0

I got the same issue (DataNode & TaskTracker would not come up).

RESOLUTION: DELETE EVERY "CURRENT" SUB-DIRECTORY UNDER: data, name, and namesecondary to resolve DataNode/taskTracker not showing when you start-all.sh, then jps

(My dfs.name.dir location is: /home/training/hadoop-temp/dfs/data/current; /home/training/hadoop-temp/dfs/name/current; /home/training/hadoop-temp/dfs/namesecondary/current

Make sure you stop services: stop-all.sh

1. Go to each "current" sub-directory under data, name, namesecondary and remove/delete (example: rm -r name/current)

2. Then format: hadoop namenode -format

3. mkdir current under /home/training/hadoop-temp/dfs/data/current

4. Take the directory and contents from /home/training/hadoop-temp/dfs/name/current and copy into the /data/current directory

EXAMPLE: files under:

/home/training/hadoop-temp/dfs/name/current

[training@CentOS current]$ ls -l

-rw-rw-r--. 1 training training 9901 Sep 25 01:50 edits

-rw-rw-r--. 1 training training 582 Sep 25 01:50 fsimage

-rw-rw-r--. 1 training training 8 Sep 25 01:50 fstime

-rw-rw-r--. 1 training training 101 Sep 25 01:50 VERSION

5. Change the storageType=NAME_NODE in VERSION to storageType=DATA_NODE in the data/current/VERSION that you just copied over.

BEFORE:

[training@CentOS dfs]$ cat data/current/VERSION

namespaceID=1018374124

cTime=0

storageType=NAME_NODE

layoutVersion=-32

AFTER:

[training@CentOS dfs]$ cat data/current/VERSION

namespaceID=1018374124

cTime=0

storageType=DATA_NODE

layoutVersion=-32

6. Make sure each subdirectory below has the same files that name/current has for data, name, namesecondary

[training@CentOS dfs]$ pwd

/home/training/hadoop-temp/dfs/

[training@CentOS dfs]$ ls -l

total 12

drwxr-xr-x. 5 training training 4096 Sep 25 01:29 data

drwxrwxr-x. 5 training training 4096 Sep 25 01:19 name

drwxrwxr-x. 5 training training 4096 Sep 25 01:29 namesecondary

7. Now start the services: start-all.sh You should see all 5 services when you type: jps

Patisserie answered 25/9, 2016 at 7:18 Comment(0)
W
0

I am using hadoop-2.6.0.I resolved using:

1.Deleting all files within /usr/local/hadoop_store/hdfs

command : sudo rm -r /usr/local/hadoop_store/hdfs/*

2.Format hadoop namenode

command : hadoop namenode -format

3.Go to ..../sbin directory(cd /usr/local/hadoop/sbin)

start-all.sh

  1. use command==> hduser@abc-3551:/$ jps

Following services would be started now :

19088 Jps
18707 ResourceManager
19043 NodeManager
18535 SecondaryNameNode
18329 DataNode
18159 NameNode
Washrag answered 9/10, 2016 at 9:15 Comment(0)
E
0

When I had this same issue, the 'Current' folder wasn't even being created in my hadoop/data/datanode folder. If this is the case for you too, ~copy the contents of 'Current' from namenode and paste it into datanode folder. ~Then, open VERSION for datanode and change the storageType=NAME_NODE to storageType=DATA_NODE ~run jps to see that the datanode continues to run

Evocator answered 15/12, 2021 at 16:4 Comment(1)
nevermind..this worked for the first time then not further more... – Evocator

© 2022 - 2024 — McMap. All rights reserved.