The Secret Life of Go: Concurrency Patterns
.jpeg)
From naive goroutines to production-grade concurrency.
Chapter 23: The WaitGroup, The ErrorGroup, and The Safety Net
"Three hundred milliseconds," Ethan sighed, staring at his dashboard metrics. "It's too slow."
"What is too slow?" Eleanor asked, pulling up a chair.
"This user profile page," Ethan pointed. "To build it, I have to fetch three things: the user's details, their recent posts, and their account stats. Each database query takes 100 milliseconds. Since I do them one after another, the user waits 300 milliseconds."
"And you want to do them all at once?"
"Exactly," Ethan said. "If I run them in parallel, it should only take as long as the slowest one—100 milliseconds."
He started typing furiously. "I'll just put the go keyword in front of each function call!"
func GetDashboard() {
go fetchUser() // 100ms
go fetchPosts() // 100ms
go fetchStats() // 100ms
// Done!
}
He ran the code. The program finished instantly. The result: Empty Screen.
"The main function didn't wait," Eleanor noted dryly. "You launched three ships, but you sailed away before they could return."
The Counter (sync.WaitGroup)
"To coordinate multiple goroutines, we need a counter," Eleanor explained. "We call it a sync.WaitGroup. It’s simple: you add to the counter before you start work, and you subtract when you finish. The main function blocks until the counter hits zero."
She refactored his code:
func GetDashboard() {
var wg sync.WaitGroup
// CRITICAL: Call Add() BEFORE launching the goroutine
// If you call it inside, the Wait() might happen before the Add()!
wg.Add(3)
go func() {
defer wg.Done() // Decrement counter when finished
fetchUser()
}()
go func() {
defer wg.Done()
fetchPosts()
}()
go func() {
defer wg.Done()
fetchStats()
}()
wg.Wait() // Block here until counter is zero
fmt.Println("All data fetched!")
}
Ethan ran it. Total time: 100ms. Success.
"But wait," Ethan frowned. "What if fetchPosts fails? WaitGroup doesn't return any errors. It just waits."
"That is the limitation," Eleanor nodded. "With a raw WaitGroup, managing errors is messy. You have to create a mutex-protected slice to collect them yourself. And worse, you can't stop the other jobs if one fails."
The Coordinator (errgroup)
"For production code, we rarely use raw WaitGroups anymore," Eleanor said. "We use the Error Group."
"Is that in the standard library?"
"It is in the extended library: golang.org/x/sync/errgroup. It combines a WaitGroup, error handling, and Context cancellation into one clean package."
She showed him the modern pattern:
import "golang.org/x/sync/errgroup"
func GetDashboard(ctx context.Context) error {
// Create an errgroup derived from the context
g, ctx := errgroup.WithContext(ctx)
g.Go(func() error {
// Pass the new context down!
// If one function fails, 'ctx' gets cancelled for everyone else.
return fetchUser(ctx)
})
g.Go(func() error {
return fetchPosts(ctx)
})
g.Go(func() error {
return fetchStats(ctx)
})
// Wait blocks until all are done.
// It returns the FIRST error that occurred (if any).
return g.Wait()
}
"This is beautiful," Ethan said. "If fetchUser fails, g.Wait() returns that error automatically?"
"Yes. And because we used WithContext, if one function fails, the errgroup automatically cancels the ctx for the other two. It stops the wasted work immediately."
The Safety Net
Ethan looked at the code, satisfied, but then a thought struck him.
"Eleanor," he asked. "In Chapter 21, we learned that a panic crashes the program. What if fetchPosts panics? Will errgroup catch it?"
"No," Eleanor warned. "That is the most dangerous trap in Go concurrency. A panic cannot cross a goroutine boundary. If one of these background tasks panics, your entire server crashes. recover() in your main function will not save you."
"So I have to put a recover() inside every goroutine?" Ethan asked, looking horrified at the boilerplate code.
"You should use a wrapper," Eleanor said. "A 'Safety Net' function."
She sketched out a helper function, a pattern often used by senior engineers.
// SafeGo wraps a function with a recovery mechanism
func SafeGo(fn func()) {
go func() {
defer func() {
if r := recover(); r != nil {
fmt.Println("Recovered from background panic:", r)
// You might also want to log this to Sentry/Datadog here
}
}()
fn()
}()
}
"Now," Eleanor said, "whenever you fire off a background task that isn't managed by a library like sourcegraph/conc (which does this for you), you use this wrapper."
func ProcessBackgroundJobs() {
// Instead of 'go process()', use:
SafeGo(func() {
process() // If this panics, only this goroutine dies. The app lives.
})
}
"So," Ethan summarized. "Use errgroup when I need to wait for results. Use SafeGo when I just want to fire and forget safely."
"Exactly," Eleanor smiled. "You are not just writing code that works now. You are writing code that survives."
Key Concepts
sync.WaitGroup
The primitive tool for waiting.
Add(n): Always call this before launching the goroutine to avoid race conditions.Done(): Call this (usually viadefer) when a task finishes.Wait(): Blocks execution until the counter hits zero.- Limitation: Does not handle errors or panic recovery.
errgroup.Group
The production-grade tool (requires golang.org/x/sync/errgroup).
- Automatic Error Propagation:
g.Wait()returns the first error encountered. - Context Integration: If initialized with
WithContext, a failure in one goroutine cancels the context for all others.
The Goroutine Boundary Rule
- A panic inside a goroutine will kill the entire process unless recovered inside that specific goroutine.
- Pattern: Use a
SafeGowrapper or a structured concurrency library likesourcegraph/concto ensure every background goroutine has adefer recover()attached to it.
Next chapter: Channels. Ethan learns how to pass data between running goroutines without fighting over memory.
Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.
Comments
Post a Comment