Checking domain name availability with DNS records

Question

How accurately can a domain name's availibility be determined by checking for the existence of NS or SOA (Start of Authority) records?

If not, how can I determine this as accurately as possible without resorting to WHOIS? For example, is it worth checking for MX records if no NS records exist?

I am specifically interested in .co.za domains, but the only official, fool-proof way to check availability in the .co.za namespace is to use http://co.za/whois.shtml, which limits consecutive checking by IP address.

Bajaj

Just because a DNS name doesn't resolve, doesn't mean it hasn't been registered. – Tania

I know, I'm trying to estimate its availability without resorting to WHOIS. – Bajaj

Answer 1

• Whois is the only 100% sure way of checking for .co.za domain availability. See http://co.za/whois.shtml
• My tests show that checking for the existence of NS records or ANY DNS records in bulk seems to be the most accurate way of guessing whether a .co.za domain is available for registration. SOA records work too, but not as well.
• dig co.za. NS shows .co.za TLD nameservers. Ping TLD servers and use the closest ones to avoid unncessary recursive queries.

Thanks for your answers.

Bajaj

Answer 2

The only 100% reliable way to check for domain availability is to query the registrar's database. I wouldn't trust whois.

You can use DNS to get an estimate. Instead of looking for an SOA record, I would just look to see if anything at all is listed with the TLD name server. dig is a good tool for this (runs on Windows, too), although I guess you could use nslookup too. For example:

dig co.za. NS

will provide a list of the name servers for .co.za. One of those servers is ns0.is.co.za.

Next, query that server directly to see if they have anything listed for your domain of interest:

dig ibm.co.za @ns0.is.co.za

That query returns NS records, but no SOA record since SOA records are provided by the domain's name server (which may or may not be online). The NS records indicate the domain name is in use.

The reason for going direct is that it's usually much faster than relying on recursive queries from your local name server.

Tableau

Hence, the existence of an NS record is a better indication that it is taken? Can you elaborate on "going direct"? – Bajaj

Yes, NS records are a better indication. If they don't exist, you can't get the SOA record or anything else meaningful using DNS. By "going direct", I mean querying only the TLD server. The normal query process goes something like this: your app queries the local DNS, which forwards to a shared DNS (optional; such as on a router) which forwards to your ISP's DNS which queries root servers to find the right TLD server, then queries the TLD server to find the domain's server, then queries the domain server. To determine domain validity, just query the TLD server; nothing else is needed. – Tableau

Is there an overarching .co.za TLD nameserver? If not, which DNS server will provide the quickest answer for .co.za queries? Bulk checking for "ANY" DNS records seems to find virtually all registered .co.za domains. – Bajaj

The first dig query above will return a list of all of the .co.za TLD servers, any one of which can handle queries for the entire domain. If you're planning a bulk query process where perf is important, you could ping all of them and see which one is topologically closest. – Tableau

Good suggestion. ns0.is.co.za. seems to have the least hops inbetween. – Bajaj

Answer 3

The only way to determine if a domain is available or not is to query the registry database for that domain. Usually you can do that by performing a WHOIS query on the registry WHOIS interface.

There are several reasons to explain why a DNS check is not safe.

The most simple explanation is because you can register a domain without pointing it to any location. It's like a house. You can buy a house, but you are not forced to build a road to reach it.

The most part of available domain providers force you to point a DNS, otherwise they set a default one. But you're not forced, it's not a kind of mandatory domain requirement.

The second reason is because there are actually cases when a domain is registered but not available. One of this cases is the period often known as redemption period. This period lasts for some days after the domain is expired and the current owner don't renew it. Several registries disable the domain, it means it is no longer reachable via DNS, but the domain still exists and can be redeemed (usually) only by the original owner.

The DNS check is cheap and sometimes faster. If you need to check whether the domain exists, then you can try a DNS query first and, in case of failure, fallback to the WHOIS query. In other words, if the DNS query succeed the WHOIS query is almost pointless. But you cannot do the opposite because if a DNS query fails, it doesn't mean the domain is not registered.

Jesselyn

Answer 4

If you plan to do bulk checks you will be soon disappointed by the built-in DNS resolver - it is synchronous. There is a nice cute library called ADNS (http://www.gnu.org/software/adns/) which offers asynchronous DNS queries. It is accompanied by a small command-line tool adnshost which I am using in combination with N-DJBDNS (http://pjp.dgplug.org/ndjbdns/) as a recursive resolver.

The exact arguments are adnshost -a -tns -f

The 4th column in the output will contain the result code:

• 0 if there is at least 1 name server assigned for the domain (most registrars assign a default one until the owner assigns his own)
• 300 or 301 if there are no name servers assigned

Checking 1.2M domain names takes about 4-6 hours but this will vary with network speed/congestion.

Infusive

Answer 5

Whois is the canonical way to check for domain availability. The rate limiting is there deliberately to keep folks from overloading the whois servers. Existence of SOA records could be a good guess, but it isn't a certain answer.

Krick

Yes, checking for SOA isn't perfect. But is there a better way, perhaps in conjunction with other records, to improve accuracy? – Bajaj

While it may be the canonical way, whois is also a notoriously unreliable way to check for domain availability, since each registrar operates their own databases, with their own non-uniform policies. – Tableau

RickNZ: it depends on the TLD. For the vast majority of the TLD, the whois server is operated by the registry, not the registrars. – Wether

Within .com, for example, there are multiple sources / databases of whois records (the so-called "thin" model), which are located with and run by registrars. Many of those servers have a reputation for being offline regularly. Also, some TLDs don't even have whois servers (including .co.za). – Tableau

Answer 6

In practice, the DNS lookup works 99% of the time. If you writing something like Ajaxwhois, I'd suggest going this route. If you have a million names to check, I would do a first pass with a DNS lookup, and then maybe do a second pass on the ones which you may consider purchasing.

Nightfall

DNS lookup fail for all the TLD where you can book a domain without publishing it in the DNS... – Wether

.co.za domains require valid nameservers upon registration. – Bajaj

Answer 7

I've also been using the dig method to test for the existence of a domain before registering but about 10% of the time, it returns domains that are NOT available as available. This is because the owner of the domain hasn't set up any records for the domain. So what I've decided to do is:

1. First check via dig method.
2. If domain appears to be available, then check using whois.shtml

People will go through a bunch of domains that are not available and this doesn't waste the allocated requests that UniForum provides. Only if something appears to be available do you check on co.za itself. It seems to me to be the only reliable way to do this WITHOUT losing access to co.za/whois.shtml due to sending them a million requests :)

Morrow

Answer 8

Tried whois and it gives me weird results (google.fr as free!) so I tried a Windows command: nslookup yourdomain and you will get its IP if it is registred.

Merce

Probably a wrong usage of whois. I just checked and, obviously, google.fr is not free. Learn to use your tools. If you want to test, goooooogle.fr is free. – Wether