Go Guide

System design · Lesson 56 of 62 · about 75 min

Design a distributed ID generator

Generate unique, roughly time-ordered 64-bit IDs on many machines without coordination: compare UUIDs, database sequences, ticket servers, Snowflake and ULID, handle clock skew, and build a Snowflake service in Go.

Twitter: Announcing Snowflake ↗RFC 9562: UUID version 7 ↗Flickr: Ticket servers ↗
On this page · 9 sections
  1. 011. Requirements
  2. 022. The options
  3. 033. Snowflake's layout
  4. 044. Deep dives
  5. 055. UUIDv7 in a few lines
  6. 06Build it: a Snowflake ID service
  7. 07Summary
  8. 08Check your understanding
  9. 09Practice in Go

1. Requirements#

  • Unique across all machines, forever.
  • 64-bit integers (fit in int64, fast to index, compact in URLs).
  • Roughly sortable by creation time.
  • High throughput: tens of thousands of IDs per second per machine, with no network call per ID.
  • Available even if other machines or a central database are down.

2. The options#

ApproachSizeSortableCoordinationNotes
database auto-increment64 bityesevery insert hits one DBsimple; a single point of failure and a bottleneck
ticket server with ranges64 bitroughlyone query per block of IDsFlickr's approach; gaps after restarts
UUIDv4 (random)128 bitnononehuge space, but random inserts hurt B-tree indexes
UUIDv7128 bityes (ms)nonetimestamp + random; the modern default when 128 bits is fine
ULID128 bityes (ms)nonelike UUIDv7 with a sortable Base32 text form
Snowflake64 bityes (ms)machine ID assigned oncetimestamp + machine + sequence

3. Snowflake's layout#

BitsFieldRange
1sign, always 0keeps the ID positive in int64
41milliseconds since a custom epochabout 69.7 years
10machine ID1,024 machines
12sequence within the millisecond4,096 IDs per ms per machine

Step through the packing with your own numbers:

Because the timestamp is in the highest bits, comparing two IDs numerically compares their creation times (to the millisecond, and assuming machines' clocks agree closely).

4. Deep dives#

  • Assigning machine IDs: from configuration (an environment variable per deployment), or claimed dynamically through a coordination service (etcd, ZooKeeper, or a database row with a unique constraint and a lease). Two machines with the same ID will produce duplicates, so this is the one thing that must be coordinated.
  • Sequence overflow: more than 4,096 IDs in one millisecond on one machine means waiting for the next millisecond. That is a cap of about 4 million IDs per second per machine.
  • Clocks going backward: NTP can step the clock back. If the generator issued IDs at time T and the clock now says T-5, it must not use T-5 again. Either wait until the clock catches up (for small jumps) or return an error and alert (for large ones).
  • Information leakage: IDs reveal creation time and roughly how many IDs a machine issues. If that matters, do not expose raw IDs publicly.

5. UUIDv7 in a few lines#

When 128 bits is acceptable, UUIDv7 needs no machine IDs at all: 48 bits of Unix milliseconds, then version and variant bits, then 74 random bits.

uuidv7.go
Go
package main
 
import (
	"crypto/rand"
	"encoding/binary"
	"fmt"
	"time"
)
 
func uuidv7(now time.Time) [16]byte {
	var u [16]byte
	rand.Read(u[:]) // random bits everywhere first
	ms := uint64(now.UnixMilli())
	binary.BigEndian.PutUint64(u[0:8], ms<<16|uint64(binary.BigEndian.Uint16(u[6:8]))) // 48-bit time in front
	u[6] = 0x70 | u[6]&0x0f                                                            // version 7
	u[8] = 0x80 | u[8]&0x3f                                                            // RFC 9562 variant
	return u
}
 
func format(u [16]byte) string {
	return fmt.Sprintf("%x-%x-%x-%x-%x", u[0:4], u[4:6], u[6:8], u[8:10], u[10:16])
}
 
func main() {
	t := time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)
	a, b := uuidv7(t), uuidv7(t.Add(time.Millisecond))
	fmt.Println("timestamp prefix:", format(a)[:13])
	fmt.Println("version:", format(a)[14:15], " later sorts after earlier:", format(b) > format(a))
	ms := binary.BigEndian.Uint64(a[0:8]) >> 16
	fmt.Println("decoded time:", time.UnixMilli(int64(ms)).UTC())
}
Output
timestamp prefix: 01a13522-4200
version: 7  later sorts after earlier: true
decoded time: 2026-10-13 12:00:00 +0000 UTC

Build it: a Snowflake ID service#

You will build a generator package that survives concurrency, sequence overflow and backward clocks, an HTTP service that issues and decodes IDs, and a page that shows each ID's bit fields.

idgen/
├── go.mod
├── main.go                         GET /api/ids?n=5, GET /api/ids/{id}
├── main_test.go
├── internal/snowflake/snowflake.go
├── internal/snowflake/snowflake_test.go
└── web/index.html

Step 1: module

Shell
mkdir idgen && cd idgen && go mod init idgen
mkdir -p internal/snowflake web
go.mod
module idgen
 
go 1.24

Step 2: the generator

internal/snowflake/snowflake.go
Go
// Package snowflake generates 64-bit, time-ordered unique IDs.
package snowflake
 
import (
	"errors"
	"sync"
	"time"
)
 
const (
	machineBits  = 10
	sequenceBits = 12
	maxMachine   = 1<<machineBits - 1
	maxSequence  = 1<<sequenceBits - 1
	// MaxBackwardWait is the largest backward clock jump the generator waits out instead of failing.
	MaxBackwardWait = 10 * time.Millisecond
)
 
// Epoch is the custom epoch (2024-01-01 UTC); 41 bits of milliseconds last until about 2093.
var Epoch = time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
 
var ErrClockBackwards = errors.New("snowflake: clock moved backwards")
 
// Generator issues IDs for one machine. It is safe for concurrent use.
type Generator struct {
	mu       sync.Mutex
	machine  int64
	lastMS   int64
	sequence int64
	now      func() time.Time
	sleep    func(time.Duration)
}
 
// New returns a generator for machineID (0-1023). now and sleep may be nil (real clock).
func New(machineID int, now func() time.Time, sleep func(time.Duration)) (*Generator, error) {
	if machineID < 0 || machineID > maxMachine {
		return nil, errors.New("snowflake: machine ID must be 0-1023")
	}
	if now == nil {
		now = time.Now
	}
	if sleep == nil {
		sleep = time.Sleep
	}
	return &Generator{machine: int64(machineID), lastMS: -1, now: now, sleep: sleep}, nil
}
 
func (g *Generator) millis() int64 { return g.now().Sub(Epoch).Milliseconds() }
 
// Next returns a new unique ID.
func (g *Generator) Next() (int64, error) {
	g.mu.Lock()
	defer g.mu.Unlock()
	ms := g.millis()
	if ms < g.lastMS {
		back := time.Duration(g.lastMS-ms) * time.Millisecond
		if back > MaxBackwardWait {
			return 0, ErrClockBackwards
		}
		g.sleep(back) // a small NTP correction: wait it out
		if ms = g.millis(); ms < g.lastMS {
			return 0, ErrClockBackwards
		}
	}
	if ms == g.lastMS {
		g.sequence = (g.sequence + 1) & maxSequence
		if g.sequence == 0 { // 4096 IDs already issued this millisecond
			for ms <= g.lastMS {
				g.sleep(time.Millisecond / 10)
				ms = g.millis()
			}
		}
	} else {
		g.sequence = 0
	}
	g.lastMS = ms
	return ms<<(machineBits+sequenceBits) | g.machine<<sequenceBits | g.sequence, nil
}
 
// Parts is a decoded ID.
type Parts struct {
	Time     time.Time `json:"time"`
	Machine  int64     `json:"machine"`
	Sequence int64     `json:"sequence"`
}
 
// Decode splits an ID back into its fields.
func Decode(id int64) Parts {
	return Parts{
		Time:     Epoch.Add(time.Duration(id>>(machineBits+sequenceBits)) * time.Millisecond).UTC(),
		Machine:  id >> sequenceBits & maxMachine,
		Sequence: id & maxSequence,
	}
}

The tests use a fake clock that only moves when the test says so, which makes overflow and backward jumps reproducible:

internal/snowflake/snowflake_test.go
Go
package snowflake
 
import (
	"errors"
	"sync"
	"testing"
	"time"
)
 
type clock struct {
	mu sync.Mutex
	t  time.Time
}
 
func (c *clock) Now() time.Time {
	c.mu.Lock()
	defer c.mu.Unlock()
	return c.t
}
 
func (c *clock) Sleep(d time.Duration) { // sleeping just advances fake time
	c.mu.Lock()
	defer c.mu.Unlock()
	c.t = c.t.Add(d)
}
 
func newTest(t *testing.T, machine int) (*Generator, *clock) {
	t.Helper()
	c := &clock{t: Epoch.Add(time.Hour)}
	g, err := New(machine, c.Now, c.Sleep)
	if err != nil {
		t.Fatal(err)
	}
	return g, c
}
 
func TestDecodeRoundTrip(t *testing.T) {
	g, c := newTest(t, 37)
	id, _ := g.Next()
	p := Decode(id)
	if p.Machine != 37 || p.Sequence != 0 || !p.Time.Equal(c.Now()) {
		t.Fatalf("Decode = %+v", p)
	}
}
 
func TestOrderedAndUniqueAcrossOverflow(t *testing.T) {
	g, _ := newTest(t, 1)
	prev := int64(-1)
	for i := range 10_000 { // more than 4096 in the same fake millisecond forces waits
		id, err := g.Next()
		if err != nil {
			t.Fatal(err)
		}
		if id <= prev {
			t.Fatalf("id %d at step %d not increasing", id, i)
		}
		prev = id
	}
}
 
func TestMachinesNeverCollide(t *testing.T) {
	g1, _ := newTest(t, 1)
	g2, _ := newTest(t, 2)
	a, _ := g1.Next()
	b, _ := g2.Next()
	if a == b {
		t.Fatal("same ID from different machines")
	}
}
 
func TestClockBackwards(t *testing.T) {
	g, c := newTest(t, 5)
	g.Next()
	c.t = c.t.Add(-3 * time.Millisecond) // small jump: waited out
	if _, err := g.Next(); err != nil {
		t.Fatalf("small backward jump: %v", err)
	}
	c.t = c.t.Add(-time.Second) // big jump: refuse
	if _, err := g.Next(); !errors.Is(err, ErrClockBackwards) {
		t.Fatalf("want ErrClockBackwards, got %v", err)
	}
}
 
func TestConcurrentUnique(t *testing.T) {
	g, err := New(9, nil, nil) // the real clock
	if err != nil {
		t.Fatal(err)
	}
	var mu sync.Mutex
	seen := map[int64]bool{}
	var wg sync.WaitGroup
	for range 8 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for range 2000 {
				id, err := g.Next()
				if err != nil {
					t.Error(err)
					return
				}
				mu.Lock()
				if seen[id] {
					t.Errorf("duplicate %d", id)
				}
				seen[id] = true
				mu.Unlock()
			}
		}()
	}
	wg.Wait()
}

Step 3: the HTTP service

IDs are returned as strings in JSON. JavaScript numbers are 64-bit floats with 53 bits of integer precision, so a raw 64-bit ID would be silently rounded in the browser. The ,string struct tag option does the conversion.

main.go
Go
package main
 
import (
	_ "embed"
	"encoding/json"
	"log/slog"
	"net/http"
	"os"
	"strconv"
	"time"
 
	"idgen/internal/snowflake"
)
 
//go:embed web/index.html
var indexHTML []byte
 
type idOut struct {
	ID int64 `json:"id,string"` // as a JSON string: JavaScript cannot hold 64-bit integers exactly
	snowflake.Parts
}
 
func newServer(g *snowflake.Generator) http.Handler {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/html; charset=utf-8")
		w.Write(indexHTML)
	})
	mux.HandleFunc("GET /api/ids", func(w http.ResponseWriter, r *http.Request) {
		n := 1
		if s := r.URL.Query().Get("n"); s != "" {
			v, err := strconv.Atoi(s)
			if err != nil || v < 1 || v > 1000 {
				http.Error(w, "n must be 1-1000", http.StatusBadRequest)
				return
			}
			n = v
		}
		out := make([]idOut, 0, n)
		for range n {
			id, err := g.Next()
			if err != nil {
				http.Error(w, err.Error(), http.StatusServiceUnavailable)
				return
			}
			out = append(out, idOut{id, snowflake.Decode(id)})
		}
		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(out)
	})
	mux.HandleFunc("GET /api/ids/{id}", func(w http.ResponseWriter, r *http.Request) {
		id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
		if err != nil || id < 0 {
			http.Error(w, "id must be a positive 64-bit integer", http.StatusBadRequest)
			return
		}
		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(idOut{id, snowflake.Decode(id)})
	})
	return mux
}
 
func main() {
	machine, err := strconv.Atoi(os.Getenv("MACHINE_ID")) // each instance needs a unique one
	if err != nil {
		machine = 0
	}
	g, err := snowflake.New(machine, nil, nil)
	if err != nil {
		slog.Error("generator", "err", err)
		os.Exit(1)
	}
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}
	slog.Info("listening", "port", port, "machine", machine)
	srv := &http.Server{Addr: ":" + port, Handler: newServer(g), ReadHeaderTimeout: 5 * time.Second}
	if err := srv.ListenAndServe(); err != nil {
		slog.Error("server", "err", err)
		os.Exit(1)
	}
}
main_test.go
Go
package main
 
import (
	"encoding/json"
	"net/http/httptest"
	"strconv"
	"testing"
	"time"
 
	"idgen/internal/snowflake"
)
 
func TestIssueAndDecode(t *testing.T) {
	now := snowflake.Epoch.Add(48 * time.Hour)
	g, _ := snowflake.New(7, func() time.Time { return now }, func(d time.Duration) { now = now.Add(d) })
	h := newServer(g)
 
	rec := httptest.NewRecorder()
	h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/ids?n=3", nil))
	var ids []struct {
		ID       string `json:"id"`
		Machine  int64  `json:"machine"`
		Sequence int64  `json:"sequence"`
	}
	if err := json.NewDecoder(rec.Body).Decode(&ids); err != nil || len(ids) != 3 {
		t.Fatalf("decode: %v, %d ids", err, len(ids))
	}
	if ids[2].Sequence != 2 || ids[0].Machine != 7 {
		t.Errorf("unexpected fields: %+v", ids)
	}
 
	rec = httptest.NewRecorder()
	h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/ids/"+ids[1].ID, nil))
	var one struct {
		ID   string    `json:"id"`
		Time time.Time `json:"time"`
	}
	json.NewDecoder(rec.Body).Decode(&one)
	if one.ID != ids[1].ID || !one.Time.Equal(now) {
		t.Errorf("decode endpoint: %+v", one)
	}
	if _, err := strconv.ParseInt(one.ID, 10, 64); err != nil {
		t.Error("id must be a decimal string")
	}
 
	for _, bad := range []string{"/api/ids?n=0", "/api/ids?n=5000", "/api/ids/abc"} {
		rec := httptest.NewRecorder()
		h.ServeHTTP(rec, httptest.NewRequest("GET", bad, nil))
		if rec.Code != 400 {
			t.Errorf("%s: status %d, want 400", bad, rec.Code)
		}
	}
}

Step 4: the page

web/index.html
HTML
<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Snowflake IDs</title>
  <style>
    body { font-family: system-ui, sans-serif; max-width: 760px; margin: 3rem auto; padding: 0 1rem; }
    table { border-collapse: collapse; width: 100%; font: 13px monospace; }
    td, th { padding: .35rem .5rem; border-bottom: 1px solid #ddd; text-align: left; }
    .bits span { padding: 0 2px; border-radius: 3px; }
    .t { background: #fbf1c7; } .m { background: #e0f1f7; } .s { background: #e1f3e8; }
    button, input { padding: .45rem .7rem; border-radius: 8px; border: 1px solid #999; }
  </style>
</head>
<body>
  <h1>Snowflake ID generator</h1>
  <p><button id="gen">Generate 5 IDs</button>
     <input id="id" placeholder="paste an ID to decode"> <button id="dec">Decode</button></p>
  <table><thead><tr><th>id</th><th>time (UTC)</th><th>machine</th><th>seq</th></tr></thead><tbody id="rows"></tbody></table>
  <p class="bits" id="bits"></p>
  <script>
    function bits(id) {
      const b = BigInt(id).toString(2).padStart(64, "0");
      return `<span class="t">${b.slice(0, 42)}</span><span class="m">${b.slice(42, 52)}</span><span class="s">${b.slice(52)}</span>`;
    }
    function show(list) {
      const tbody = document.getElementById("rows");
      for (const x of list) {
        const tr = document.createElement("tr");
        for (const v of [x.id, x.time, x.machine, x.sequence]) {
          const td = document.createElement("td");
          td.textContent = v;
          tr.append(td);
        }
        tr.onclick = () => document.getElementById("bits").innerHTML = bits(x.id);
        tbody.prepend(tr);
      }
      if (list.length) document.getElementById("bits").innerHTML = bits(list[list.length - 1].id);
    }
    document.getElementById("gen").onclick = async () => show(await (await fetch("/api/ids?n=5")).json());
    document.getElementById("dec").onclick = async () => {
      const res = await fetch("/api/ids/" + encodeURIComponent(document.getElementById("id").value.trim()));
      if (res.ok) show([await res.json()]);
      else alert(await res.text());
    };
  </script>
</body>
</html>

BigInt in the page parses the string ID exactly, so the bit display is correct for all 64 bits. Click a row to see its timestamp (yellow), machine (blue) and sequence (green) bits.

Step 5: run several machines

Shell
go test -race ./...
MACHINE_ID=1 PORT=8081 go run . &
MACHINE_ID=2 PORT=8082 go run . &
curl -s 'localhost:8081/api/ids?n=2'; echo
curl -s 'localhost:8082/api/ids?n=2'; echo

The two instances issue different IDs in the same millisecond because their machine bits differ. In production, assign MACHINE_ID per replica (a Kubernetes StatefulSet ordinal works well) or claim it from a database with a unique constraint and a heartbeat.

Summary#

  • Choose by needs: auto-increment for one database, UUIDv7 when 128 bits is fine, Snowflake for compact sortable 64-bit IDs.
  • Snowflake packs 41 bits of time, 10 of machine and 12 of sequence; only the machine ID needs coordination.
  • Wait on sequence overflow, refuse or wait on backward clocks, and never let two machines share an ID.
  • Send 64-bit IDs to browsers as strings.
  • The build tests every edge case with a fake clock that the test controls.

Check your understanding

Answer before scrolling back up. If you miss one, reread the matching section, then try again.

  1. Q1.Why are random UUIDv4 primary keys often slow in B-tree indexes?

  2. Q2.In a Snowflake ID, what guarantees two machines never produce the same ID in the same millisecond?

  3. Q3.What should a Snowflake generator do when its 12-bit sequence overflows within one millisecond?

  4. Q4.The system clock jumps backward by 5 ms (an NTP correction). A safe generator:

  5. Q5.How does a ticket-server range allocator reduce database load?

Practice in Go

Write a table test first, then the solution. On LeetCode, choose Go as the language. Open a hint only after 15 minutes stuck.

  • Snowflake with datacenter bitsbuild

    Split the 10 machine bits into 5 datacenter + 5 worker bits. Add both to the decode endpoint and the page.

  • Machine IDs from leasesbuild

    Instead of a MACHINE_ID environment variable, claim a free machine ID by inserting into a machines table with a unique constraint and a heartbeat column, and release stale claims after 1 minute.

  • UUIDv7 packagebuild

    Turn the UUIDv7 function from this lesson into a package with Parse, String and Time methods, and fuzz-test Parse.

Finished this lesson?

Mark it done to track your progress. It is saved in this browser only.