rabbitmq-server fails to start after hostname has changed for first time

Question

I am using django-celery for my django project. Last day I have changed my computer's hostname (I am using Ubuntu 12.04, edited file '/etc/hostname'), and after next restart django-celery was failing with error

Consumer: Connection Error: [Errno 111] Connection refused. Trying again in 4 seconds...

After some research on this error I could find that, changing my host name caused this error from here. My rabbitmq startup log shows

file: /var/log/rabbitmq/startup_log

Activating RabbitMQ plugins ...

********************************************************************************
********************************************************************************

0 plugins activated:

ERROR: epmd error for host "jinesh": nxdomain (non-existing domain)

My startup_err file is empty.

when I run

root@jinesh:/home/jinesh# rabbitmqctl list_users
Listing users ...
Error: unable to connect to node rabbit@jinesh: nodedown

DIAGNOSTICS
===========

nodes in question: [rabbit@jinesh]

hosts, their running nodes and ports:
- unable to connect to epmd on jinesh: nxdomain

current node details:
- node name: rabbitmqctl4956@jinesh
- home dir: /var/lib/rabbitmq
- cookie hash: RGhmB2JR1LbZ57j7xWWTxg==

I hope changing the nodename may fix this issue. But I couldn't found a way to do this. Anyone have idea about how solve this issue?

update

while changing hostname you have to change both /etc/hostname and /etc/hosts files.

I reinstalled rabbitmq and solved this issue, Will answer this question.

Bruges

On centos/rhel, if you rename your network interfaces via /etc/udev/rules/70-persistant-net.rules and reboot --> doing so will also cause you to get an error message from rabbitmqctl status "unable to connect to epmd". --> The fix is the same as the accepted answer by @RichardHFung for this question. – Headstone

"systemctl restart rabbitmq-server.service" solved the same issue I had. – Longlimbed

If on Ubuntu 16.04 or 18.04, do NOT install RabbitMQ via the repository (they're outdated). Head here rabbitmq.com/install-debian.html and save yourself some headache down the line, by installing both Erlang and RabbitMQ from one of the proposed repositories. I had a while ago installed Erlang from erlang solutions and had much later installed RabbitMQ via its Ubuntu repo and it originally worked without fuss. After a recent upgrade something went wrong and I spent 5 hours trying to fix it to no avail. I removed both packages and reinstalled Bintray's versions as explained. Fixed. – Britney

Answer 1

Remove the old installation of RabbitMQ to fix this problem. Here are steps to reinstall RabbitMQ. These commands are run as the root user:

1. Stop RabbitMQ: rabbitmqctl stop

2. Change /etc/hosts

3. Change /etc/hostname

4. Uninstall old RabbitMQ: dpkg -P rabbitmq-server

5. Remove RabbitMQ’s database: rm -rf /var/lib/rabbitmq

6. Find erlang’s process that is running rabbit: ps ax | grep rabbit

7. Kill the listed process

8. Reinstall RabbitMQ: apt-get install rabbitmq-server

I wrote about these steps on my blog.

REVISION

I moved my blog to a new website.

Quartana

Note that I didn't have to stop rabbit (because it wasn't running) or remove the /var/lib/rabbitmq folder (because it wasn't there); YMMV. – Purse

on ubuntu (14.04.1 LTS) this did not work for me. I needed to apt-get purge rabbitmq-server and then run apt-get install rabbitmq-server to get it all working again after a hostname change. – Selfpreservation

RabbitMQ breaks for no good reason, and the only way to fix it is to uninstall and reinstall?This is alarming... – Vendetta

Neither the manual instructions nor the apt-get instructions worked for me. As of yet, still no solution. Is this an erlang problem? – Fellner

Anyway of doing this without losing data? – Hoecake

my problem was that there was an erlang process running, killed it and rabbit started as normal. – Flews

i gave downvote because of solution is not a real solution. what if i change hostname again? remove rabbit again, install again .... – Solberg

Check @Kishor Pawar answer if you don't want to lose data – Finsen

Answer 2

Thanks to Richard H Fung.

His steps helped me to solve this issue.

But I did not have to re-install the rabbitmq.

When I opened my /etc/hosts file I found that IP assigned to my hostname is different than the actual ip(192.168.1.200 [static]).

#/etc/hosts  
127.0.0.1       localhost  
192.168.1.115   HOSTNAME

so I just changed IP address to 192.168.1.200 in my /etc/hosts file and it worked fine.

Chalcography

this is the exact situation i had and same solution worked - no need to reinstall and fixed ip and worked – Erumpent

This should be accepted as the correct solution. On xenial I had /etc/hosts associating a GloballyRoutableIP with the hostname name (not fqdn), and found that associating the NetworkTenIP with that name repaired epmd. Killing epmd and restarting rabbit sufficed. – Childbearing

Phew! Didn't have to re-install either. My issue after changing the hostname was that I don't use a FQDN in /etc/hostname. This is OK, but you then need to put that short alias in /etc/hosts for 127.0.0.1 or the erlang system can't resolve it. I did however loose all queue contents, presumably that store is associated with the old hostname. Possibly this could be solved by putting an IP address in the conf file... – Ascertain

Answer 3

Richard's answer is good, but you might lose some information in the rabbitmq queues. The following is a possible way to preserve the previous setup of rabbitmq with the new host name.

A Short Answer:

If you want to keep the new host name change, then you can create a rabbitmq-env.conf files in /etc/rabbitmq that references the old host name and all should be good. The following is what should be in the rabbitmq-env.conf file:

NODENAME=rabbit@OLDHOSTNAME

After adding the config file, restart the rabbitmq server then you should be good. (e.g. service rabbitmq-server restart (might need a sudo with ubuntu))

For more details you can read from the rabbitmq website: https://www.rabbitmq.com/man/rabbitmq-env.conf.5.man.html

More Details About the Answer:

I had a similar hostname issues using CentOS... The root issue was when rabbit installs, it references the current host name at the time of install. Since this is a rabbitmq thing, the solution should work for other linux flavors. If you want to see the full answer, you can see it at: rabbtimqadmin - Could not connect: [Errno -2] Name or service not known

Shadwell

This helped me. I updated to the latest version of rabbitmq (3.6.6-1) on ArchLinux and was unable to start it. Took a look in /etc/rabbitmq/rabbitmq-env.conf and 'lo and behold, the NODENAME was all wrong. Instead of rabbit@machine, it had become rabbit@machine@machine during the upgrade. A simple removal of the last @machine fixed the problem. – Curet

This is the proper answer IMHO...@Chewtoy it's even in the Arch wiki :-D wiki.archlinux.org/index.php/Rabbitmq#Changed_hostname – Russellrusset

So awesome!. Thank you. I changed my hostname and I didn't wanted to lose any information (usernames, etc). The env.conf file fixed my problem. – Goutweed

Where is this file on Windows? – Kalil

Answer 4

You could solve this problem by either deleting the erlang mnesia database associated with rabbitmq or reinstalling rabbitmq.

I got hint for first solution from rabbitmq mailing list. Excerpt from answer, The Erlang Mnesia database is host specific (because it is a distributed DB). The simplest way to get you fixed is to clear out the database dir.

The second method is the easiest way (not recommend though). To uninstall do

dpkg -P rabbitmq-server

You can refer this link if you would like to know more about installing/removing debian packages.

Bruges

How do you find the Erlang Mnesia Database? – Rambo

Hi @MatthewCanty: I followed the second method. – Bruges

@MattCanty At rabbitmq 3.6.9-1, just rm -rf /var/lib/rabbitmq/mnesia/. And the Erlang Mnesia Database is under the file path of /var/lib/rabbitmq/mnesia/. – Locally

Answer 5

On Windows, the issue I was facing was due to the McAfee firewall. The exact error was:

epmd error for host "<HOSTNAME>": address (cannot connect to host/port)

It started working as soon as I disabled the firewall

Northington

This helped me. been straggling with that for 2 hours, Disabled it and worked like magic, Thanks! – Donny

Answer 6

Remove RabbitMQ database: rm -rf /var/lib/rabbitmq/*

This action solves the problem. I believe somewhere in dumps stored a record with correspondence of nodes host names and ips. This fact causes contradiction if some hosts changed unexpectedly.

Apportion

Be careful while doing this. You will lose all data – Lewanna

Answer 7

In my case I did not have a following entry in /etc/hosts:

127.0.0.1 <hostname>

where <hostname> is the hostname of my machine (as given by the hostname command). After adding that line, RabbitMQ started successfully. I did not have any earlier installations of RabbitMQ.

Reptilian

For me, this was the answer which worked. However I'm still puzzled why the default 127.0.0.1 localhost.localadmin localhost line didn't resolve to the appropriate nodename. Thank you very much! – Anteroom

Answer 8

My solution was to check and correct the $HOSTNAME and /etc/hostname. It turned out that my router was a little confused and gave me a wrong hostname domain. After restarting this one, I called export HOSTNAME=the.correct.hostname and my rabbit runs without restarting the server os.

Phalarope

It worked, without even uninstalling rabbitmq-server – Cardamom

Answer 9

When on Windows (sorry about that - I understand the topic is for Ubuntu) just reinstall the RabbitMQ service:

rabbitmq-service.bat remove
rabbitmq-service.bat install

The script is located at C:\Program Files\RabbitMQ Server\rabbitmq_server-<version>\sbin folder.

Or may be just changing the value of HKEY_LOCAL_MACHINE\SOFTWARE\Ericsson\Erlang\ErlSrv\1.1\RabbitMQ\@SName is enough.

Clues: https://github.com/rabbitmq/rabbitmq-server/issues/620

Joinville

Answer 10

To solve this problem, I changed my /etc/hostname file with the new hostname then restarted the machine. After that, I ran rm -rf /var/lib/rabbitmq/mnesia/*

Then restarted the service : sudo service rabbitmq-server restart

At this point, it worked for me. If it still don't work, modify the file /etc/rabbitmq/rabbitmq-env.conf by adding those informations :

NODENAME=rabbit@YOUR_NEW_HOSTNAME
NODE_IP_ADDRESS=127.0.0.1
NODE_PORT=5672

Then restart the service : sudo service rabbitmq-server restart

Hope it helps.

Vicenary

I had similiar problem, rabbitmq had issues after i changed the hostname. And this worked for me. Thanks – Araldo

Answer 11

I had the same problem and doing the steps Richard H Fung provided didn't helped me. I also had to uninstall these packages: erlang erlang-epmd rabbitmq-server

zypper remove erlang erlang-epmd rabbitmq-server
rm -rf /var/lib/rabbitmq/*
zypper install erlang erlang-epmd rabbitmq-server
systemctl enable rabbitmq-server.service
systemctl start rabbitmq-server.service

If you have another OS than openSUSE just use your package manager to remove these packages (e.g. apt-get)

Epa

Answer 12

1. Change to old hostname.

   • sudo hostnamectl set-hostname (old_hostname)

2. Start rabbitmq-server and stop again.

   • sudo systemctl start rabbitmq-server.service
   • sudo systemctl stop rabbitmq-server.service

3. Change to new hostname.

   • sudo hostnamectl set-hostname (new_hostname)

4. Start rabbitmq-server.

   • sudo systemctl start rabbitmq-server.service

Tricky

Answer 13

The only solution which work for me: install erlang & rabbitmq from deb, so:

First remove:

apt-get purge rabbitmq-server
apt-get purge erlang
apt-get autoremove
reboot

After install wget:

sudo apt-get -y install socat logrotate init-system-helpers adduser
sudo apt-get -y install wget

Install erlang:

wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
sudo dpkg -i erlang-solutions_1.0_all.deb
sudo apt-get update
sudo apt-get install erlang

Install rabbitmq:

sudo apt-get update
wget https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.7.17/rabbitmq-server_3.7.17-1_all.deb
sudo dpkg -i rabbitmq-server_3.7.17-1_all.deb
rm rabbitmq-server_3.7.17-1_all.deb

Oster

Answer 14

Check qpidd daemon, it could be also already hogging the port that rabbitmq needs.

sudo netstat -lnp

Kulseth