Mac OSX Lion DNS lookup order [closed]

Question

After upgrading to Mac OSX Lion I figured out that /etc/hosts is not looked up in first place for name resolution anymore. This leads to some side effects like:

1. Entries in /etc/hosts are resolved painfully slow
2. You can't not override existing domains, e.g. 127.0.0.1 www.google.com
3. If you get search domain entries from DHCP, let say .lan, and some funny guy configured localhost.lan to something else then 127.0.0.1 in the local DNS you can not reach your localhost anymore.

Is this behavior intended? Does it make any sense? And most important, how can I come back to the old behavior.

Pergola

Super helpful question - surprise, surprise its closed as off topic – Limeade

At least they've not deleted the thread.. yet. This saved my bacon. I changed all my hosts from X.local to X.lhost and problem gone. On a side note I'm a big fan of xip.io e.g. foo.127.0.0.1.xip.io – Incomprehension

Answer 1

I think he matter is Lion handles .local TLD differently because it's reserved for some Multicast DNS features (used by Bonjour). The only way i found to solve this issue is using a different TLD for development hosts (ie: .dev). It works fine for me, hope it's gonna be helpful to others!

Caning

My first thought was "lame". However, then I stumbled on this other stack post and changed my stance: serverfault.com/questions/17255/… – Desmond

One note - if you use chrome for development, non-standard top level domains will be interpreted as search. You may need to do something like .dev.com to make it do an actual domain lookup. I'm not sure how to do this elegantly. – Atypical

@bbrame: You can enter your local domain with the url scheme: http://foo.dev/ ; After that, Chrome will realize that foo.dev is a domain and not a query. – Sourdine

Alternatively you can use dscl tool to add an exception. – Sesquialtera

.local was never slow for me up until two weeks ago. Switching to .dev worked amazing though!! I've had Lion on this laptop for nearly two years now... so it's odd that it suddenly just stopped working (especially seeing that this is an old issue) – Philanthropist

Answer 2

With regards to overriding domains in the hosts file, I have found that in some circumstances, Lion queries the IPv6 address for a domain if it senses that a domain is unreachable over the IPv4 network.

I discovered this when I noticed some ads that I had never seen before on Snow Leopard because I had redirected the ad domains to 127.0.0.1. I fired up wireshark and noticed AAAA (IPv6 DNS records) queries following the IPv4 A queries (IPv4). The ad servers indeed have IPv6 addesses and were able to serve me their content.

The solution to this is have a

::1 mydomain.com

entry for every

127.0.0.1 mydomain.com

entry in your hosts file.

Interestingly, if you happen to have a local webserver running on 127.0.0.1:80 and your browser receives a response from the webserver (error or otherwise), no AAAA query is issued, as it seems to be satisfied that a TCP connection was at least possible.

---

On a related note, if you make heavy use of the hosts file (for adblocking, local web development, etc), you may want to look into running your own local DNS resolver. There is a considerable disk/CPU hit from having to read /etc/hosts on every request, so it is in your best interest to keep that file very light.

One advantage of running something like dnsmasq locally (besides the significant performance boost) is that you can redirect whole top-level domains back to your local machine. This allows you to have the whole *.dev namespace for development (for instance), without having to individually enter every domain you want resolved locally into /etc/hosts

Sourdine

Thanks so much for this. Waiting 10-30 seconds to test changes to my code was driving me crazy and you saved me a ton of time by not having to figure this out myself. – Kattiekatuscha

I had the same issue, and this fixed my problem immediately! Nice. – Galatia

+1 This is a great tidbit for anyone searching for "why does my hosts file not work". I may ask that question here just so you can put the same answer there and make it easier to find via a search engine! – Soundless

There should be no appreciable increase in disk I/O from reading /etc/hosts - the OS will cache the file if it's used frequently. – Boot

Users whose LANs support IPv6 (it's almost 2016, after all!) will encounter this problem from now until IPv4 is completely gone....or until Apple picks up on the problem and solves it internally! Jean-Baptiste's response should also be considered (i.e., use .dev instead of .local for your dev environments). – Folie

Answer 3

The problem was that I symlinked the /etc/hosts file. If /etc/hosts is a plain file everything is ok.

Pergola

I'm having the same problem. However, my /etc/hosts file is a normal file. Any help on this would be appreciated. – Delirious

This appears to have been my issue as well. I had a symlink to a file in my dropbox folder, which used to work and which I thought was mighty clever. It would appear that Apple no longer finds this clever. I also did a full real restart using Option-restart after moving it from a symlink to a real file. Everything seems happy now. – Jair

@Scott: You will be able to comment on arbitrary answers after collecting 50 reputation points. – Guardian

this is exactly my problem, but I don't understand why it would be a problem, symlinks were fine in snow leopard. Very frustrating. – Highborn

Entries in a symlinked hosts file are ok if they can't otherwise be resolved, this indicates that a symlinked hosts file is only checked when an address can't otherwise be resolved. When the hosts file is a normal file it's checked before any other form of resolution. So if you need to override domains that actually have valid DNS entries, your hosts file must be a file and not a symlink. – Highborn

For the record, this is still the case in Mavericks (10.9), would be useful if someone could confirm what Yosemite does… – Hock

Yosemite does it as well, just ran into this problem. This is extremely odd behavior. – Masquer

This is still the behaviour in High Sierra. – Bechler

Answer 4

Update(2): OSX 10.10.5 brings the return of mDNSResponder.

Update: OSX 10.10 Yosemite has replaced mDNSResponder with "discoveryd". I've not upgraded so I am not sure of the discoveryd behavior w/r/t DNS lookups and /etc/hosts.

The system DNS resolver on Lion is the mDNSResponder process.

You may be thinking "but mDNSResponder is the multicast dns responder." You're right; that's what it originally was for, and it still fulfills this function. However, on newer MacOS versions it also does standard host lookups.

In Lion, it does not appear to automatically re-read /etc/hosts when it changes, at least not always. Killing mDNSResponder (and allowing it to be automatically restarted) seems to fix the problem.

sudo killall mDNSResponder

should do the trick.

below is my original answer for posterity. I suppose it might still be an issue in some cases.

Make sure your /etc/hosts file is a unix style text file, with linefeeds as the ending rather than cr's.

Editing with TextWrangler or a unix text editor should preserve the file.

If your file is already messed up, try this to fix

tr '\015' '\012' < /etc/hosts > /tmp/hosts.$$
mv /etc/hosts /etc/hosts.bad
mv /tmp/hosts.$$ /etc/hosts
# fix up permissions while we are at it
chown root:wheel /etc/hosts
chmod 644 /etc/hosts

credit for this fix to:

http://techpatio.com/2011/guides-how-to/fixed-mac-osx-lion-etc-hosts-bugs-dns

Boot

This might solve a problem with a crashed DNS resolver daemon, but it does not solve the problem of resolving hosts to LAN IP addresses often encountered in the dev environments. @Sourdine response, below, will be the right solution for most people finding this question in search; although Jean-Baptiste-MONIN has an answer with merits, too. – Folie

It can solve the problem of changes to /etc/hosts not being noticed. – Boot

I'm using high sierra and this answer resolve the alias problem, thanks – Presbytery

Answer 5

ive had this issue for a while, as im working a team of devs it became necessary to actually use .local rather then .dev or .localhost, i found this article to be very useful.

iTand.me - Lion local domains and etc hosts..

In summary;

But if you have to use .local, the most elegant solution I've found is the dscl utility. Using it is very straightforward. To add a host called mydev.local and point it to the localhost, just do this:

sudo dscl localhost -create /Local/Default/Hosts/mydev.local IPAddress 127.0.0.1

To see all the currently defined hosts and their IPs

sudo dscl localhost -list /Local/Default/Hosts IPAddress

And to remove a host:

sudo dscl localhost -delete /Local/Default/Hosts/mydev.local

Overall, pretty straightforward and works well. I still would prefer to be able to edit /etc/hosts instead, but this is a better alternative to having to rename all our .local servers.

Comma

When adding a hostname this way, it seemingly does not do anything. Cannot ping the address. Example: sudo dscl localhost -create /Local/Default/Hosts/test1 IPAddress 127.0.0.1 ping test1 ping: cannot resolve test1: Unknown host – Hoitytoity

Answer 6

Prior to moving from Snow Leopard to Lion, I had several app-specific entries in /etc/hosts, like this:

127.0.0.1 foo.bar.local

After the update, loading my local apps was VERY slow. I noticed that the delay happened before the request showed up in the log file, and that once it did, the app itself was as fast as usual.

Now I have two lines per app, like this:

127.0.0.1 foo.bar.local
::1       foo.bar.local

... and everything is fast again.

Apparently this adds IPv6 addresses? I don't quite get it, really, but it works.

Brandon

Nothing else worked for me but this did in an instant - thanks Nathan! – Cervicitis

Answer 7

My situation was similar, but the delays, of exactly 5 seconds, only happened for URLs ending with '.local'. When viewing sites that ended in '.dev', there was no delay.

Some of the other developers in my office had this problem, while a few did not. I was hoping for a simple fix and I did not want to rename the site to '.local' due to other dependencies.

I ran the following command in Terminal and diffed my output with a few other users in the office.

scutil --dns

This section was the only difference:

resolver #2
  domain   : 00000000.members.btmm.icloud.com
  options  : pdns
  timeout  : 5
  order    : 150000

My Mac was linked to my iCloud account and I had Back To My Mac enabled. Once I disabled Back To My Mac, the additional resolver went away and the 5 second delay disappeared.

Phonate

Answer 8

Wow, what a nightmare. I have read absolutely everything on this subject and everything that has been suggested so far was tantilizingly close to what I was experiencing, but none of the solutions worked for me.

And I figured out why.

Unlike others, I was not using /etc/hosts to set up local domains. My /etc/hosts file was stock, containing only the entries needed for the loopback interface and the broadcast host. Moreover, it was a correctly-encoded unix file, as I'm the sort of person who would only edit that from the command-line using emacs. And, thank goodness, I did not have to resort to running my own DNS server like DNSmasq to get around the problem.

(To be clear, the symptom that brought me here to this issue was that emacs took about 10 seconds to start, but only when I was on wifi. If I turned off wifi, emacs would start up instantly as expected.)

My solution: my laptop has a name, "terminator". (Yes, its shiny aluminum exterior made me think of the Arnold Schwarzenegger character.) I just needed to add entries to /etc/hosts for the name of the machine itself:

127.0.0.1   terminator
::1         terminator

I found the name of my host by running a simple command in the terminal:

hostname

...which came back with the output: "terminator". After changing /etc/hosts to contain those two entries, emacs can now quickly resolve my laptop's name.

I hope this helps someone.

Decarbonize

this seems to have worked for me just now. We will see if this holds. I am THRILLED you figured this out, because I have intermittently seen this problem crop up w/o warning. – Harmful

Shoot. This is not a permanent resolution for me. Problem back. Mind you, as I re-read this, my issue is not yours... – Harmful

Answer 9

I've had speed issues using OSX Lion as a web development box ... Using a combination of suggestions I resorted to disabling ipv6 networking and routing ipv6 to localhost6 ... things sped up quite a bit ...

sudo networksetup -setv6off Ethernet

/etc/hosts ...

127.0.0.1    localhost
127.0.0.1    dev.aliasdomain.com
... 
::1          localhost6 

Uppercase

Answer 10

I think there's been some bug fixes. I've seen a lot of problems mentioned, and none of these seem to apply currently (for example, putting multiple aliases on a single line now works fine for me).

At any rate it seems that with Lion, Apple made some drastic changes to mDNSResponder which handles all the DNS lookups, and (with Lion at least) also handles /etc/hosts cacheing. For me forward lookups also now work. But reverse lookups (e.g. looking up 1.2.3.4 instead of google.com) don't work.

After a lot of pain, it looks like mDNSResponder converts this lookup to 4.3.2.1.in-addr.arpa and does a name lookup. This may well be how DNS prefers to operate, but it doesn't work at all with /etc/hosts.

Unless of course you add an alias of 4.3.2.1.in-addr.arpa for each host, where 4.3.2.1 is the ip address in the opposite order from which you are used to seeing it. This fixes everything for me. Here's an example /etc/hosts entry:

1.2.3.4 foo foo.example.com alias.example.com 4.3.2.1.in-addr.arpa

Peaslee