Varnish Sharding Cluster

Varnish Sharding Cluster

A working demo of a three-node Varnish cluster with consistent object sharding and self-routing, built on vmod_directors and a hash-based shard director. Each node runs identical VCL — no external load balancer, no coordinator. The shard director determines object ownership deterministically: a request for /index.html always maps to the same node, regardless of which node receives it.

If the owning node is down, the director falls back to the next node in the ring. Cluster topology changes (node up/down) are handled automatically via health checks.

→ github.com/pawboss/varnish-sharding-cluster

              CLIENT
           ┌────┬────┬────┐
        :8091  :8092  :8093
           │    │    │
           ▼    ▼    ▼
        ┌──────────────────┐
        │ Varnish 1/2/3    │  ← same VCL, shard director
        │                  │
        │  vcl_recv:       │
        │  hash(req.url)   │
        │  → pick owner    │
        └────────┬─────────┘
                 │  hit?  serve from local cache
                 │  miss? route to owning node (peer backend)
                 │  owner down? next node in ring (fallback)
                 │
                 ▼
          ┌────────────┐
          │   Apache   │  ← origin, only reached on true miss
          └────────────┘

Each object lives in exactly one node's cache. Cross-node requests are backend-to-backend, transparent to the client. The origin only gets hit on a true cache miss — once warmed, the cluster is self-contained.

Stack: Varnish · VCL · vmod_directors · Docker Compose · Apache