DP, math & design · Lesson 42 of 62 · about 65 min
Designing data structures
Combine maps, linked lists, heaps and slices into custom structures: LRU and LFU caches, a priority task scheduler, an O(1) randomized set and a time-versioned key-value store, all safe for concurrent use.
On this page · 9 sections
The method#
- Write down every operation and its required cost.
- For each operation, ask which structure answers it: "find by key" → map; "oldest / newest" → linked list or deque; "smallest / highest priority" → heap; "random element" or "by position" → slice; "sorted range" → sorted slice or balanced tree.
- Decide what the structures store about each other (a map value that points to a list node, or to a slice index) so that every operation can update all of them in its time budget.
- Write the invariants down, then make every method preserve them.
LRU cache#
"Least recently used": with room for capacity entries, evict the entry that was used longest ago.
| Operation | Needed | Structure |
|---|---|---|
| get(key) | O(1) lookup | map from key to list node |
| mark as most recent | O(1) move | doubly linked list, front = newest |
| evict least recent | O(1) remove from the back | the same list |
The linked lists lesson used container/list. Writing the list yourself with generics and a sentinel node (a dummy that is both before the first and after the last element) removes every nil check:
package main
import (
"fmt"
"strings"
"sync"
)
type entry[K comparable, V any] struct {
key K
val V
prev, next *entry[K, V]
}
// LRU is a fixed-capacity cache, safe for concurrent use.
type LRU[K comparable, V any] struct {
mu sync.Mutex
cap int
items map[K]*entry[K, V]
head entry[K, V] // sentinel: head.next is the most recent, head.prev the least
}
func NewLRU[K comparable, V any](capacity int) *LRU[K, V] {
c := &LRU[K, V]{cap: capacity, items: make(map[K]*entry[K, V], capacity)}
c.head.next, c.head.prev = &c.head, &c.head // an empty circle
return c
}
func (c *LRU[K, V]) unlink(e *entry[K, V]) { e.prev.next, e.next.prev = e.next, e.prev }
func (c *LRU[K, V]) pushFront(e *entry[K, V]) {
e.prev, e.next = &c.head, c.head.next
c.head.next.prev = e
c.head.next = e
}
func (c *LRU[K, V]) Get(k K) (V, bool) {
c.mu.Lock()
defer c.mu.Unlock()
e, ok := c.items[k]
ifok {
var zero V
return zero, false
}
c.unlink(e)
c.pushFront(e)
return e.val, true
}
func (c *LRU[K, V]) Put(k K, v V) (evicted K, didEvict bool) {
c.mu.Lock()
defer c.mu.Unlock()
if e, ok := c.items[k]; ok {
e.val = v
c.unlink(e)
c.pushFront(e)
return evicted, false
}
e := &entry[K, V]{key: k, val: v}
c.items[k] = e
c.pushFront(e)
if len(c.items) > c.cap {
last := c.head.prev
c.unlink(last)
delete(c.items, last.key)
return last.key, true
}
return evicted, false
}
func (c *LRU[K, V]) String() string {
var parts []string
for e := c.head.next; e != &c.head; e = e.next {
parts = append(parts, fmt.Sprintf("%v=%v", e.key, e.val))
}
return "[" + strings.Join(parts, " ") + "] (most recent first)"
}
func main() {
c := NewLRU[string, int](2)
c.Put("a", 1)
c.Put("b", 2)
fmt.Println(c)
v, ok := c.Get("a")
fmt.Println("get a:", v, ok, c)
k, ev := c.Put("c", 3)
fmt.Println("put c evicts", k, ev, c)
_, ok = c.Get("b")
fmt.Println("get b:", ok)
var wg sync.WaitGroup
shared := NewLRU[int, int](100)
for g := range 8 {
wg.Add(1)
go func() {
defer wg.Done()
for i := range 1000 {
shared.Put(g*1000+i, i)
shared.Get(i)
}
}()
}
wg.Wait()
fmt.Println("after 8 goroutines, entries:", len(shared.items))
}[b=2 a=1] (most recent first)
get a: 1 true [a=1 b=2] (most recent first)
put c evicts b true [c=3 a=1] (most recent first)
get b: false
after 8 goroutines, entries: 100The sentinel trick: with head acting as a node that is always present, inserting at the front and unlinking from anywhere never need "is this the first or last node?" branches. The sync.Mutex makes it safe for concurrent use; run the program with go run -race . to confirm there are no data races.
LFU cache#
"Least frequently used": evict the entry with the lowest use count; among ties, the least recently used one. All operations O(1):
items: key → node (value and frequency);buckets: frequency → a recency list of the keys with exactly that frequency (front = most recent);minFreq: the lowest frequency currently present.
On every access, move the key from bucket f to bucket f+1. If bucket f was the minimum and is now empty, minFreq becomes f+1. A new key always has frequency 1, so inserting resets minFreq to 1. Eviction takes the back of buckets[minFreq]. Frequencies only ever change by one, which is why minFreq never needs a search.
package main
import (
"container/list"
"fmt"
)
type lfuItem struct {
key, val, freq int
elem *list.Element // position inside buckets[freq]
}
type LFU struct {
cap, minFreq int
items map[int]*lfuItem
buckets map[int]*list.List // freq -> keys, front = most recent
}
func NewLFU(capacity int) *LFU {
return &LFU{cap: capacity, items: map[int]*lfuItem{}, buckets: map[int]*list.List{}}
}
func (c *LFU) touch(it *lfuItem) {
old := c.buckets[it.freq]
old.Remove(it.elem)
if old.Len() == 0 {
delete(c.buckets, it.freq)
if c.minFreq == it.freq {
c.minFreq++ // the lowest bucket emptied; this key is now exactly one higher
}
}
it.freq++
if c.buckets[it.freq] == nil {
c.buckets[it.freq] = list.New()
}
it.elem = c.buckets[it.freq].PushFront(it.key)
}
func (c *LFU) Get(k int) int {
it, ok := c.items[k]
ifok {
return -1
}
c.touch(it)
return it.val
}
func (c *LFU) Put(k, v int) {
if c.cap == 0 {
return
}
if it, ok := c.items[k]; ok {
it.val = v
c.touch(it)
return
}
if len(c.items) == c.cap {
b := c.buckets[c.minFreq]
victim := b.Remove(b.Back()).(int) // least recent among the least frequent
if b.Len() == 0 {
delete(c.buckets, c.minFreq)
}
delete(c.items, victim)
fmt.Printf(" evict %d (freq %d)\n", victim, c.minFreq)
}
if c.buckets[1] == nil {
c.buckets[1] = list.New()
}
c.items[k] = &lfuItem{key: k, val: v, freq: 1, elem: c.buckets[1].PushFront(k)}
c.minFreq = 1
}
func main() {
c := NewLFU(2)
c.Put(1, 10)
c.Put(2, 20)
fmt.Println("get 1:", c.Get(1)) // key 1 now has freq 2
fmt.Println("put 3:")
c.Put(3, 30) // evicts 2 (freq 1)
fmt.Println("get 2:", c.Get(2), " get 3:", c.Get(3))
fmt.Println("put 4:")
c.Put(4, 40) // 1 and 3 both have freq 2; 1 was used less recently
fmt.Println("get 1:", c.Get(1), " get 3:", c.Get(3), " get 4:", c.Get(4))
}get 1: 10
put 3:
evict 2 (freq 1)
get 2: -1 get 3: 30
put 4:
evict 1 (freq 2)
get 1: -1 get 3: 30 get 4: 40A task scheduler with priorities#
A job runner must always start the highest-priority ready task, and among equal priorities the one submitted first (FIFO fairness). A heap ordered by (priority desc, sequence asc) does exactly that, with O(log n) submit and next. Two extensions make it realistic:
- Delayed tasks: keep them in a second heap ordered by
runAt; before picking work, move every task whose time has come into the ready heap. - Aging: a low-priority task that waited too long gets its effective priority raised, so a flood of urgent work cannot starve it forever.
package main
import (
"container/heap"
"fmt"
)
type Task struct {
Name string
Priority int // higher runs first
RunAt int // earliest time it may start
seq int // submission order, for FIFO among equal priorities
}
type readyQ []*Task
func (q readyQ) Len() int { return len(q) }
func (q readyQ) Less(i, j int) bool {
if q[i].Priority != q[j].Priority {
return q[i].Priority > q[j].Priority
}
return q[i].seq < q[j].seq
}
func (q readyQ) Swap(i, j int) { q[i], q[j] = q[j], q[i] }
func (q *readyQ) Push(x any) { *q = append(*q, x.(*Task)) }
func (q *readyQ) Pop() any {
old := *q
t := old[len(old)-1]
*q = old[:len(old)-1]
return t
}
type delayQ []*Task
func (q delayQ) Len() int { return len(q) }
func (q delayQ) Less(i, j int) bool { return q[i].RunAt < q[j].RunAt }
func (q delayQ) Swap(i, j int) { q[i], q[j] = q[j], q[i] }
func (q *delayQ) Push(x any) { *q = append(*q, x.(*Task)) }
func (q *delayQ) Pop() any {
old := *q
t := old[len(old)-1]
*q = old[:len(old)-1]
return t
}
type Scheduler struct {
ready readyQ
delayed delayQ
seq int
}
func (s *Scheduler) Submit(t *Task) {
s.seq++
t.seq = s.seq
heap.Push(&s.delayed, t)
}
// Next returns the task to run at time now, or nil if nothing is ready.
func (s *Scheduler) Next(now int) *Task {
for s.delayed.Len() > 0 s.delayed[0].RunAt <= now {
heap.Push(&s.ready, heap.Pop(&s.delayed)) // its time has come
}
if s.ready.Len() == 0 {
return nil
}
return heap.Pop(&s.ready).(*Task)
}
func main() {
var s Scheduler
s.Submit(&Task{Name: "send newsletter", Priority: 1})
s.Submit(&Task{Name: "resize images", Priority: 5})
s.Submit(&Task{Name: "charge card", Priority: 9})
s.Submit(&Task{Name: "nightly report", Priority: 9, RunAt: 3})
s.Submit(&Task{Name: "thumbnail", Priority: 5})
for now := 0; now < 6; now++ {
if t := s.Next(now); t != nil {
fmt.Printf("t=%d run %-15s (priority %d)\n", now, t.Name, t.Priority)
} else {
fmt.Printf("t=%d idle\n", now)
}
}
}t=0 run charge card (priority 9)
t=1 run resize images (priority 5)
t=2 run thumbnail (priority 5)
t=3 run nightly report (priority 9)
t=4 run send newsletter (priority 1)
t=5 idleAt t=3 the delayed "nightly report" becomes ready and, with priority 9, jumps ahead of "send newsletter", which has been waiting since t=0. The two priority-5 tasks run in submission order thanks to seq. In a real service, Next would block on a sync.Cond or a channel instead of returning nil, and workers would call it in a loop.
RandomizedSet: O(1) insert, delete and random pick#
| Operation | Structure |
|---|---|
| insert, delete, contains | map from value to its index in a slice |
| uniform random element | a slice: pick a random index |
Deleting from the middle of a slice is O(n), unless order does not matter: then swap the element with the last one, truncate, and update the moved element's index in the map: O(1).
Time-based key-value store#
Set(key, value, timestamp) and Get(key, timestamp), which returns the value with the largest timestamp <= t. Timestamps arrive in increasing order, so per key keep a slice of (timestamp, value) pairs, which is automatically sorted, and binary search it. This is the core of versioned configuration, feature flags with history, and multi-version databases (MVCC).
package main
import (
"fmt"
"math/rand/v2"
"sort"
)
type RandomizedSet struct {
vals []int
idx map[int]int
rng *rand.Rand
}
func NewRandomizedSet() *RandomizedSet {
return &RandomizedSet{idx: map[int]int{}, rng: rand.New(rand.NewPCG(1, 2))}
}
func (s *RandomizedSet) Insert(v int) bool {
if _, ok := s.idx[v]; ok {
return false
}
s.idx[v] = len(s.vals)
s.vals = append(s.vals, v)
return true
}
func (s *RandomizedSet) Remove(v int) bool {
i, ok := s.idx[v]
ifok {
return false
}
last := s.vals[len(s.vals)-1]
s.vals[i] = last // move the last element into the hole
s.idx[last] = i
s.vals = s.vals[:len(s.vals)-1]
delete(s.idx, v)
return true
}
func (s *RandomizedSet) Random() int { return s.vals[s.rng.IntN(len(s.vals))] }
type versioned struct {
ts int
val string
}
type TimeMap map[string][]versioned
func (m TimeMap) Set(k, v string, ts int) { m[k] = append(m[k], versioned{ts, v}) }
func (m TimeMap) Get(k string, ts int) string {
h := m[k]
i := sort.Search(len(h), func(i int) bool { return h[i].ts > ts }) // first entry after ts
if i == 0 {
return ""
}
return h[i-1].val
}
func main() {
s := NewRandomizedSet()
fmt.Println(s.Insert(10), s.Insert(20), s.Insert(30), s.Insert(20))
fmt.Println(s.Remove(10), s.vals, s.idx)
counts := map[int]int{}
for range 6000 {
counts[s.Random()]++
}
fmt.Println("roughly even picks:", counts[20] > 2700 counts[30] > 2700)
tm := TimeMap{}
tm.Set("price", "100", 1)
tm.Set("price", "120", 5)
tm.Set("price", "90", 9)
fmt.Printf("%q %q %q %q\n", tm.Get("price", 0), tm.Get("price", 4), tm.Get("price", 5), tm.Get("price", 100))
}true true true false
true [30 20] map[20:1 30:0]
roughly even picks: true
"" "100" "120" "90"Summary#
- Start from the operations and their costs; give each operation a structure; connect the structures with stored pointers or indexes.
- LRU: map + doubly linked list (a sentinel node removes edge cases). LFU: map + frequency buckets +
minFreq. - Priority schedulers are heaps keyed by (priority, sequence), with a second heap for delayed work and aging against starvation.
- O(1) delete from a slice when order does not matter: swap with the last and truncate.
- Guard shared structures with a mutex, and test with
-race.
Check your understanding
Answer before scrolling back up. If you miss one, reread the matching section, then try again.
Q1.An LRU cache needs O(1) get and put. Which pair of structures gives that?
Q2.In an LFU cache, which entry is evicted?
Q3.How does the LFU cache find the minimum frequency in O(1)?
Q4.A RandomizedSet removes an element in O(1) from a slice by:
Q5.Why might a priority scheduler add 'aging'?
Practice in Go
Write a table test first, then the solution. On LeetCode, choose Go as the language. Open a hint only after 15 minutes stuck.
- LRU Cache ↗medium
Show hint
Map from key to list node; move to front on access; evict from the back.
- LFU Cache ↗hard
Show hint
Map key -> node, map freq -> list, and a minFreq integer.
Show hint
Slice of values plus map value -> index; delete by swapping with the last.
Show hint
Per key, a slice of (timestamp, value) in increasing order; binary search for the last <= t.
- Task Scheduler ↗medium
Show hint
Greedy: (maxCount-1)*(n+1) + number of tasks with maxCount, or simulate with a heap.
- Design Twitter ↗medium
Show hint
Per-user tweet lists plus a heap merge of the followed users' latest tweets.
- Thread-safe LRU for godsabuild
Make the LRU cache generic (
LRU[K comparable, V any]), guard it with async.Mutex, add a-racetest with 8 goroutines, and benchmark Get against a plain map.
Finished this lesson?
Mark it done to track your progress. It is saved in this browser only.