Go Guide

Go, chapter by chapter · Lesson 8 of 62 · about 50 min

Core packages

A guided tour of the standard library you will use every week: strings, io, files and folders, errors, containers, sorting and hashing.

On this page · 12 sections
  1. 01strings
  2. 02io: Readers and Writers
  3. 03Files and folders
  4. 04errors
  5. 05Containers and sorting
  6. 06Hashes and cryptography
  7. 07Fast input for algorithm practice
  8. 08More examples
  9. 09The full program
  10. 10Summary
  11. 11Check your understanding
  12. 12Practice in Go

strings#

The strings package works on strings without changing them (strings are immutable); every function returns a new value.

FunctionExampleResult
Contains(s, sub)Contains("gopher", "ph")true
Index(s, sub)Index("gopher", "ph")2 (or -1)
HasPrefix / HasSuffixHasPrefix("golang", "go")true
Split(s, sep)Split("a,b,c", ",")[a b c]
Fields(s)Fields(" go is fun ")[go is fun] (splits on any whitespace)
Join(parts, sep)Join([]string{"a","b"}, "-")a-b
Repeat(s, n)Repeat("ab", 3)ababab
Replace(s, old, new, n)Replace("aaa", "a", "b", 2)bba (-1 means all)
ToLower / ToUpperToUpper("go")GO
TrimSpace(s)TrimSpace(" hi\n")hi

To build a string piece by piece, use strings.Builder. Adding with += in a loop copies the whole string each time (O(n²)); a Builder grows a buffer like append does (O(n)). Its sibling, strconv, converts between strings and numbers (Atoi, Itoa, ParseFloat, FormatInt).

io: Readers and Writers#

Two tiny interfaces carry most data in Go:

Go
type Reader interface { Read(p []byte) (n int, err error) }
type Writer interface { Write(p []byte) (n int, err error) }

Files, network connections, HTTP bodies, compressed streams, strings.NewReader and bytes.Buffer all implement one or both. So a function written against io.Reader works with all of them, and your tests can pass a string instead of a real file.

  • io.Copy(dst, src) streams everything from a Reader to a Writer.
  • io.ReadAll(r) reads everything into a []byte (fine for small inputs).
  • bytes.Buffer is an in-memory Reader and Writer at once.
  • bufio adds buffering: bufio.NewScanner(r) reads line by line; bufio.NewReader and bufio.NewWriter batch reads and writes.

Files and folders#

TaskUse
Read a whole small filedata, err := os.ReadFile("a.txt")
Write a whole fileos.WriteFile("a.txt", data, 0o644)
Stream a large filef, err := os.Open(...), defer f.Close(), then bufio.NewScanner(f)
Create or appendos.Create, or os.OpenFile with flags
List a folderentries, err := os.ReadDir(".")
Walk a treefilepath.WalkDir(root, fn)
Build paths portablyfilepath.Join("data", "a.txt")

0o644 is a Unix permission: owner can read and write, everyone else can read.

errors#

errors.New("out of stock") makes a simple error value. Declare it once at package level when callers need to check for it:

Go
var ErrNoStock = errors.New("out of stock")

You will learn wrapping with %w, errors.Is and errors.As in the "Go since the book" lesson.

Containers and sorting#

The container packages give ready-made structures:

PackageStructureTypical use
container/listdoubly linked listLRU caches
container/heapheap algorithms over your slicepriority queues, Dijkstra
container/ringcircular listround-robin

For sorting:

  • slices.Sort(s) sorts any slice of ordered values (newest, fastest, generic).
  • slices.SortFunc(s, cmp) sorts with your comparison function.
  • sort.Slice(s, less) is the older reflection-based version you will see in existing code.
  • sort.Sort(x) sorts any type implementing sort.Interface (Len, Less, Swap).

Hashes and cryptography#

A hash function turns any data into a fixed-size fingerprint.

  • hash/crc32 and hash/fnv are fast, non-cryptographic: good for checksums and hash tables.
  • crypto/sha256 is cryptographic: use it when someone might try to forge data.
  • SHA-1 and MD5 are broken for security; avoid them for anything new.
  • Never store passwords with a plain hash; use golang.org/x/crypto/bcrypt or argon2.

Fast input for algorithm practice#

Competitive programming judges feed large inputs through stdin. Unbuffered fmt.Scan can be the reason a correct solution times out. The standard template:

Go
in := bufio.NewReader(os.Stdin)
out := bufio.NewWriter(os.Stdout)
defer out.Flush() // forgetting this prints nothing
 
var n int
fmt.Fscan(in, &n)

More examples#

Files and folders, hands on

The program works inside a temporary directory so it never touches your real files. os.WriteFile and os.ReadFile cover most needs; os.Create plus a bufio.Writer is for writing piece by piece; filepath.WalkDir visits a whole tree.

files.go
Go
package main
 
import (
	"bufio"
	"errors"
	"fmt"
	"io/fs"
	"os"
	"path/filepath"
	"strings"
)
 
func main() {
	dir, err := os.MkdirTemp("", "golesson-")
	if err != nil {
		panic(err)
	}
	defer os.RemoveAll(dir) // clean up everything at the end
 
	// Whole-file write and read.
	notes := filepath.Join(dir, "notes.txt")
	if err := os.WriteFile(notes, []byte("line one\nline two\nline three\n"), 0o644); err != nil {
		panic(err)
	}
	data, _ := os.ReadFile(notes)
	fmt.Printf("read %d bytes\n", len(data))
 
	// Line by line with a Scanner.
	f, _ := os.Open(notes)
	sc := bufio.NewScanner(f)
	for n := 1; sc.Scan(); n++ {
		fmt.Printf("%d: %s\n", n, sc.Text())
	}
	f.Close()
 
	// Build a small tree, then walk it.
	os.MkdirAll(filepath.Join(dir, "src", "util"), 0o755)
	for _, name := range []string{"src/main.go", "src/util/strings.go", "src/util/README.md"} {
		os.WriteFile(filepath.Join(dir, name), []byte("x"), 0o644)
	}
	filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return err
		}
		rel, _ := filepath.Rel(dir, path)
		if strings.HasSuffix(path, ".go") {
			fmt.Println("go file:", filepath.ToSlash(rel))
		}
		return nil
	})
 
	// A missing file is an error you can test for.
	_, err = os.ReadFile(filepath.Join(dir, "missing.txt"))
	fmt.Println("missing file is fs.ErrNotExist:", errors.Is(err, fs.ErrNotExist))
}
Output
read 29 bytes
1: line one
2: line two
3: line three
go file: src/main.go
go file: src/util/strings.go
missing file is fs.ErrNotExist: true

errors.Is(err, fs.ErrNotExist) works through wrapping, which is why it is preferred over comparing error strings.

Sorting the book's people

The book sorts a slice of Person by name and then by age using sort.Sort with a custom type. The modern way is slices.SortFunc with a comparison function; both are shown so you can read older code.

sorting.go
Go
package main
 
import (
	"cmp"
	"fmt"
	"slices"
	"sort"
)
 
type Person struct {
	Name string
	Age  int
}
 
// ByName implements sort.Interface, the pre-generics way.
type ByName []Person
 
func (p ByName) Len() int           { return len(p) }
func (p ByName) Less(i, j int) bool { return p[i].Name < p[j].Name }
func (p ByName) Swap(i, j int)      { p[i], p[j] = p[j], p[i] }
 
func main() {
	kids := []Person{{"Jill", 9}, {"Jack", 10}, {"Ana", 9}}
 
	sort.Sort(ByName(kids))
	fmt.Println("by name (sort.Sort):", kids)
 
	slices.SortStableFunc(kids, func(a, b Person) int { return cmp.Compare(a.Age, b.Age) })
	fmt.Println("by age, stable:     ", kids) // Ana before Jill: their name order is kept
 
	i, found := slices.BinarySearchFunc(kids, 10, func(p Person, age int) int { return cmp.Compare(p.Age, age) })
	fmt.Println("first age-10 kid:", kids[i].Name, found)
}
Output
by name (sort.Sort): [{Ana 9} {Jack 10} {Jill 9}]
by age, stable:      [{Ana 9} {Jill 9} {Jack 10}]
first age-10 kid: Jack true

Hashes: checksums and fingerprints

The book compares two files with crc32 and introduces cryptographic hashes with sha1. Today, use CRC32 (or xxhash) to detect accidental corruption quickly, and SHA-256 whenever someone could tamper with the data on purpose. SHA-1 is broken for security uses.

hashes.go
Go
package main
 
import (
	"crypto/sha256"
	"encoding/hex"
	"fmt"
	"hash/crc32"
	"io"
	"strings"
)
 
func checksum(r io.Reader) uint32 {
	h := crc32.NewIEEE()
	io.Copy(h, r) // a hash is an io.Writer: stream data into it
	return h.Sum32()
}
 
func main() {
	a := checksum(strings.NewReader("hello world"))
	b := checksum(strings.NewReader("hello world"))
	c := checksum(strings.NewReader("hello World"))
	fmt.Println("same content, same crc:", a == b, " one letter changed:", a == c)
 
	sum := sha256.Sum256([]byte("password123"))
	fmt.Println("sha256:", hex.EncodeToString(sum[:])[:16]+"…")
	fmt.Println("digest size:", len(sum), "bytes")
}
Output
same content, same crc: true  one letter changed: false
sha256: ef92b778bafe771e…
digest size: 32 bytes

The full program#

packages.go
Go
package main
 
import (
	"bufio"
	"bytes"
	"container/list"
	"crypto/sha256"
	"errors"
	"fmt"
	"hash/crc32"
	"io"
	"io/fs"
	"os"
	"path/filepath"
	"slices"
	"strconv"
	"strings"
)
 
var ErrNoStock = errors.New("out of stock")
 
// countWords works with any Reader: files, network or strings.
func countWords(r io.Reader) (map[string]int, error) {
	counts := map[string]int{}
	sc := bufio.NewScanner(r)
	sc.Split(bufio.ScanWords)
	for sc.Scan() {
		counts[strings.ToLower(sc.Text())]++
	}
	return counts, sc.Err()
}
 
func main() {
	// strings and strconv
	fmt.Println(strings.Fields(" go  is fun "), strings.Split("a,b,c", ","),
		strings.Repeat("ab", 3), strings.Index("gopher", "ph"))
	var sb strings.Builder
	for i := range 3 {
		sb.WriteString(strconv.Itoa(i))
	}
	fmt.Println(sb.String())
 
	// io
	var buf bytes.Buffer
	io.Copy(&buf, strings.NewReader("Go go GO stop"))
	counts, _ := countWords(&buf)
	fmt.Println(counts["go"], counts["stop"])
 
	// files and folders
	dir, _ := os.MkdirTemp("", "demo")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "notes.txt")
	if err := os.WriteFile(path, []byte("line 1\nline 2\n"), 0o644); err != nil {
		fmt.Println(err)
		return
	}
	f, err := os.Open(path)
	if err != nil {
		fmt.Println(err)
		return
	}
	defer f.Close()
	lines := bufio.NewScanner(f)
	for lines.Scan() {
		fmt.Println("read:", lines.Text())
	}
	filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
		if err == nil && !d.IsDir() {
			info, _ := d.Info()
			fmt.Println("file:", d.Name(), info.Size(), "bytes")
		}
		return nil
	})
	_, err = os.ReadFile(filepath.Join(dir, "missing.txt"))
	fmt.Println("missing file is an error:", err != nil)
 
	// errors
	fmt.Println(ErrNoStock)
 
	// containers and sorting
	l := list.New()
	l.PushBack("b")
	l.PushFront("a")
	l.PushBack("c")
	for e := l.Front(); e != nil; e = e.Next() {
		fmt.Print(e.Value, " ")
	}
	fmt.Println()
	nums := []int{5, 2, 9, 1}
	slices.Sort(nums)
	fmt.Println(nums)
 
	// hashes
	fmt.Println(crc32.ChecksumIEEE([]byte("gopher")))
	fmt.Printf("%x\n", sha256.Sum256([]byte("gopher")))
}
Output
[go is fun] [a b c] ababab 2
012
3 1
read: line 1
read: line 2
file: notes.txt 14 bytes
missing file is an error: true
out of stock
a b c 
[1 2 5 9]
875285291
9cc1ee455a3363ffc504f40006f70d0c8276648a5d3eb3f9524e94d1b7a83aef

Summary#

  • strings, strconv and strings.Builder cover text processing.
  • io.Reader and io.Writer connect files, networks and buffers; write functions against them.
  • os.ReadFile, os.WriteFile, os.Open with bufio.Scanner, os.ReadDir and filepath.WalkDir handle files and folders.
  • container/list, container/heap and slices.Sort cover common data structures.
  • Use CRC32 for checksums, SHA-256 for integrity, and bcrypt for passwords.

Check your understanding

Answer before scrolling back up. If you miss one, reread the matching section, then try again.

  1. Q1.Why do so many standard library functions accept an io.Reader instead of a file?

  2. Q2.What is the modern replacement for ioutil.ReadFile?

  3. Q3.Why wrap os.Stdin in bufio.NewReader when reading a million numbers?

  4. Q4.Which hash should you use to store something security-sensitive?

Practice in Go

Write a table test first, then the solution. On LeetCode, choose Go as the language. Open a hint only after 15 minutes stuck.

  • Word frequency CLIbuild

    Read a text file, count words case-insensitively with strings.Fields and a map, sort by count, and print the top 10.

    Show hint

    Write count(r io.Reader) map[string]int so your test can pass strings.NewReader("a b a").

  • Fast I/O benchmarkbuild

    Generate a file with 1,000,000 numbers. Read it with fmt.Scan and with bufio + fmt.Fscan, and time both.

    Show hint

    Use time.Since(start). Expect the buffered version to be many times faster.

  • Folder sizebuild

    Use filepath.WalkDir to total the size of every file under a folder and list the 5 largest.

    Show hint

    d.Info() returns fs.FileInfo, which has Size().

  • Show hint

    Count letters of the magazine in a [26]int, then subtract for the note.

  • Show hint

    XOR all bytes of both strings, or compare letter counts.

Finished this lesson?

Mark it done to track your progress. It is saved in this browser only.