Home/Go/Goroutines, channels, and worker pools

Goroutines, channels, and worker pools

Bounded parallelism, indexed results, channel ownership and cancellation

A goroutine is a concurrently scheduled function invocation. Starting one with go does not wait for it, give it a useful result path, or propagate its panic to the caller. A parent operation needs an explicit lifetime: start work, communicate results/errors, cancel unnecessary work, and wait for every goroutine it owns. Concurrency can improve throughput when work waits or uses multiple cores, but creating one goroutine per unbounded input can exhaust memory or overload a dependency.

go
ready := make(chan struct{})
go func() { close(ready) }()
<-ready // an explicit completion event

Channels carry both data and synchronization

An unbuffered send waits for a receiver; a buffered channel permits bounded queued values. Capacity changes backpressure, not ownership. A receive from a closed channel drains queued values then returns zero with ok=false. Sending to a closed channel panics. The producer that knows no more values will be sent normally owns closing the channel; receivers should not close a channel shared with active senders. Nil-channel sends/receives block forever, which can deliberately disable a select case but also hide initialization bugs.

A worker pool sends job indices through a channel and lets a fixed number of workers process them. Indices preserve output order even when completion order differs. Workers write distinct preallocated output positions, and the parent waits before reading them. A WaitGroup joins lifetimes; Add must happen before the corresponding work can call Done, and Wait does not replace synchronization around truly shared mutable state.

Every blocking boundary needs an exit

If a receiver exits after one error while producers keep sending, goroutines can leak permanently. Use a derived context, select on cancellation when sending jobs, and stop workers promptly on cancellation. A transform must cooperate with its supplied context; the pool cannot forcibly terminate arbitrary blocking code. Wait for already-started transforms before returning so the caller knows its operation is over.

An error channel needs an ownership and draining policy too. A buffered single-error slot can retain a first failure, or a mutex can protect one stored error. Preserve a transform's error category, cancel sibling work, and discard partial output under this exercise's all-or-error contract. The first observed concurrent error may depend on scheduling, so a deterministic API should not promise input-order error precedence unless it deliberately coordinates that.

Test concurrency with channels, barriers and explicit counts rather than sleeps. A timeout is only a guard against a hung test; it must not be used to decide correct output timing. The final tests prove that multiple callbacks can enter concurrently, never exceed the worker bound, preserve result order, and exit on cancellation. Run the same suite with the race detector to examine the exercised interleavings.

Put it into practice

Implement the exercise in a full-screen workspace, then run its tests.

Open practice
Next lessonContext, cancellation, and channel lifetimes