Ahosting Logo
Knowledge Base

Setting Up and Maintaining a DNS Cluster

The cluster failure that is quietChange made in WHM or cPanelpushed to the other cluster membersChange made in a zone file by handnot pushed: the cluster does not watch the filesystemThe other nameservers keep answeringwith records that are now out of dateCheck each nameserver with a lookup. Drift is silent, and your clients find it before you do.

A DNS cluster keeps zone records synchronised across several servers, so a domain answers from more than one machine. It is what makes proper nameserver redundancy possible when you run more than one server.

The concept is simple. The failure mode: servers that stop agreeing, is quiet, and that is what makes it worth understanding properly.

Why bother

Two reasons, and only one of them is about redundancy.

Redundancy. If one nameserver is unreachable, resolvers try another. Without a cluster, both your nameservers are the same machine, and "two nameservers" is a formality rather than protection.

Consistency. When accounts live on several servers, a cluster means a zone edited anywhere is correct everywhere. Without it, each server holds its own idea of the zone.

The roles

Standalone. No clustering. The default.

Synchronize: changes flow both ways. Both servers hold every zone.

Write-only. This server pushes changes out and does not accept them back.

Synchronise is the usual choice for two servers acting as each other's backup. Write-only suits a setup where one machine is authoritative and others only receive.

Mixing roles inconsistently. One server set to synchronise and its partner set to write-only, produces changes that propagate in one direction and silently not the other. Set both ends deliberately.

Setting it up

In WHM, open DNS Cluster. Add the other server by hostname, with its access hash for authentication, and choose the role.

Then do the same on the other server, pointing back. A cluster configured on one side only does not work, and the symptom is changes that appear on one machine and never on the other.

Enable DNS clustering in the server's configuration as well, adding a server to the list is not sufficient on its own.

The access hash is a root credential

Worth stating plainly: the access hash authenticates as root to the other server's API.

Treat it as you would a root password. Do not put it in a script, do not paste it into a ticket, and regenerate it if it has been exposed.

Clustering also requires the servers to reach each other on the API port, which means a firewall rule on both sides. Allow the specific addresses rather than opening the port generally.

Synchronisation is not automatic in the way people expect

This is the detail that produces most cluster problems.

New zones and changes made through WHM or cPanel are pushed to cluster members. A zone edited directly in a file on the server is not. The cluster does not watch the filesystem.

So a manual edit to a zone file lives on one server only, and the other keeps serving the old records. Which server a visitor asks decides what they get, and the result is a domain that works intermittently for no visible reason.

Make zone changes through the interface. If you must edit directly, synchronise the zone afterwards.

Forcing a sync

WHM offers a synchronise action for the cluster, and there is a command-line equivalent for a single zone.

Run it after any direct edit, after adding a server to an existing cluster, and whenever you suspect drift. Syncing a zone that is already correct costs nothing.

For a new cluster member, an initial full sync is required. It does not receive existing zones by joining, only the changes that happen afterwards.

Checking that both servers agree

The check that catches drift, and the only one that matters:

dig @ns1.example.com example.com A +short
dig @ns2.example.com example.com A +short

Ask each nameserver directly and compare. If they differ, the cluster is not doing its job for that zone.

Do this after any significant change, and periodically. Drift accumulates silently, and the report you eventually get ("the site works for some people") is one of the harder ones to interpret without this check. The DNS troubleshooting guide goes into querying properly.

Removing a server

Remove it from the cluster configuration on every remaining member, not only from the one you are looking at.

A half-removed member still receives pushes from servers that were not updated, and can still answer queries with data nobody is maintaining.

Also update the domain's nameserver records if the removed server was one of them: a nameserver listed at the registrar but no longer in the cluster serves stale answers indefinitely.

Private nameservers on top

A cluster is the machinery; branded nameservers are what clients see. The two are set up separately, and both are needed for a proper reseller arrangement. Setting up private nameservers deals with the visible half.

Make sure each nameserver's glue record points at the right server. A cluster synchronising perfectly still fails if the registry sends resolvers to the wrong address.

Before you migrate accounts

If you are moving accounts between servers, the cluster affects the order of operations.

A zone existing on both servers means either can answer during the move, which is helpful. It also means a stale zone on the old server can keep answering after the move, sending visitors to a site that is no longer there.

Remove the zone from the old server once the move is confirmed. Transferring cPanel accounts goes over the rest of the sequence.

Verify both directions actually work

A cluster relationship is configured on each server separately, and one side working does not imply the other does.

whmapi1 listtrustedusers 2>/dev/null | head
dnsadmin --status 2>/dev/null
for ns in ns1.example.com ns2.example.com; do
  printf '%-22s %s\n' "$ns" "$(dig "@$ns" example.com SOA +short | head -1)"
done

Querying each nameserver directly for the same zone is the test that matters. Identical answers mean the zone reached both. A difference means synchronisation is not happening, whatever the configuration screen reports.

Run that check after adding a server rather than assuming, and again after any change to the trust configuration. A cluster that silently stopped synchronising looks correct until a zone edit fails to propagate.

Watch for zones that exist on one side only

The failure that produces intermittent resolution is a zone present on one nameserver and absent from another.

for ns in ns1.example.com ns2.example.com; do
  printf '%-22s ' "$ns"; dig "@$ns" example.com NS +short | wc -l
done

A nameserver returning nothing for a domain it should serve answers queries with a refusal, and roughly half of visitors get that answer.

The symptom is a site that works for some people and not others, with no pattern anybody can find, because it depends on which nameserver each resolver happened to ask. Checking every nameserver individually is the only way to see it. Secondary DNS covers why the redundancy is worth having.