From Fork to Framework, Part 4: Why Go Won the Backend

A goroutine is the single design primitive that convinced us to rebuild two production backends in Go after shipping the same system three times in three different languages. In practice, a goroutine is a lightweight, independently executing function managed entirely by the Go runtime rather than by the operating system. Because each one starts at roughly 2 kilobytes of stack memory instead of the 1 megabyte or more an OS thread requires, a single Go process can juggle hundreds of thousands of them without breaking a sweat. This guide explains exactly what a goroutine is, how Go's scheduler runs them, how to create and synchronize them safely, and — because theory only gets you so far — what happened when we replaced thousands of Ruby processes with goroutines in a live LLM gateway.

What Is a Goroutine?

A goroutine is a function that runs concurrently with other functions in the same address space, scheduled by Go's own runtime instead of the operating system kernel. In other words, when you write go doSomething(), you are not creating an OS thread. You are handing a function to Go's scheduler, which decides when and where it runs. Consequently, goroutines are often called "green threads" or "lightweight threads," though that undersells how different the economics actually are.

For comparison, a single OS thread typically reserves around 1 megabyte of stack space, and a full process — such as a Ruby Puma worker — reserves roughly 150 megabytes once the interpreter, loaded gems, and framework are accounted for. A goroutine, by contrast, starts at about 2 kilobytes and grows or shrinks dynamically as needed. As a result, spinning up ten thousand goroutines costs less memory than spinning up a dozen OS threads.

Goroutines vs. Threads: Why Size Matters

This size difference is not a minor footnote — it changes what kinds of systems are even feasible to build. Imagine a gateway holding open a thousand simultaneous streaming connections while tokens flow from an upstream provider to a downstream client. In a thread-per-connection or process-per-connection model, that workload requires a thousand heavyweight workers. In Go, it requires a thousand goroutines occupying roughly 2 megabytes combined. That gap — on the order of 75,000x — is the reason Go's concurrency model gets singled out for high-throughput network services.

How the Go Scheduler Runs Thousands of Goroutines

Behind every goroutine sits Go's M:N scheduler, which multiplexes many goroutines (the "N") across a much smaller number of OS threads (the "M"). Specifically, the runtime maintains logical processors — represented internally as P structures — that pair OS threads with runnable goroutines. When a goroutine blocks on network input/output, which is the dominant operation in a streaming proxy, the runtime parks it instantly and schedules a different goroutine on that same OS thread. There is no Global Interpreter Lock standing in the way, and there is no hard thread-per-request ceiling.

GOMAXPROCS and the M:N Threading Model

GOMAXPROCS controls how many OS threads can execute Go code simultaneously, and it defaults to the number of logical CPU cores available. In addition, the scheduler uses a work-stealing algorithm: when one logical processor's run queue empties, it steals goroutines from a busier processor's queue rather than sitting idle. This keeps CPU cores saturated without any manual load-balancing code. For deeper detail on the scheduler's internals, the official Go documentation on goroutines is the authoritative reference, and the broader language context is covered well on Wikipedia's Go programming language entry.

Creating Your First Goroutine in Go

Starting a goroutine requires nothing more than the go keyword placed before a function call. Because the call returns immediately, the calling function continues executing while the goroutine runs independently in the background.

func fetchToken(id int, wg *sync.WaitGroup) {
    defer wg.Done()
    // simulate a network call to an upstream LLM provider
    time.Sleep(50 * time.Millisecond)
    fmt.Printf("token batch %d streamed\n", id)
}

func main() {
    var wg sync.WaitGroup
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go fetchToken(i, &wg)
    }
    wg.Wait()
}

In this example, a thousand goroutines are launched to simulate a thousand concurrent streaming connections. Specifically, each goroutine costs roughly 2 kilobytes of stack memory, so the entire batch consumes a few megabytes total — a workload that would otherwise demand a thousand OS threads or, in a process-per-connection model like Rails' Puma, roughly 150 gigabytes of RAM.

A Simple Goroutine Example Without WaitGroup

It's worth noting that launching a goroutine without any synchronization mechanism is a common beginner mistake. If main() returns before the goroutine finishes, the program exits and the goroutine's work is simply discarded. Therefore, every goroutine that needs to complete before your program moves on requires some form of synchronization — covered next.

How Goroutines Communicate: Channels and Select

Goroutines rarely work in isolation; they need to pass data back and forth safely. Go's answer is the channel, a typed conduit created with make(chan T) that lets one goroutine send a value while another receives it. Because channel operations block until both sides are ready, channels double as a synchronization tool and a data pipe at the same time — a philosophy often summarized as "share memory by communicating, not communicate by sharing memory."

Buffered vs. Unbuffered Channels

• Unbuffered channels force the sending goroutine to wait until a receiver is ready, which guarantees a handoff happens before execution continues.
• Buffered channels, created with make(chan T, n), let a sender push up to n values without blocking, which smooths out bursty producer-consumer workloads.
• The select statement lets a single goroutine wait on multiple channel operations simultaneously, choosing whichever is ready first — essential for timeouts and cancellation signals.

For a deeper reference on channel semantics and the broader synchronization primitives available, the official sync package documentation is an excellent companion to this guide.

Synchronizing Goroutines with WaitGroup and Mutex

Two primitives handle most synchronization needs. First, sync.WaitGroup lets the calling goroutine wait for a set of other goroutines to finish, as shown in the earlier example. Second, sync.Mutex protects shared state from simultaneous access by multiple goroutines, preventing the classic "two goroutines write the same variable at once" bug.

In practice, well-designed Go programs minimize shared mutable state and lean on channels instead. However, when shared state is unavoidable — such as an in-memory counter or cache — a mutex is the correct tool, and reaching for one is far cheaper than debugging a race condition in production at 2 a.m.

Common Goroutine Pitfalls (and How to Avoid Them)

Goroutines are cheap, but they are not free, and misusing them creates bugs that are notoriously hard to reproduce. The two most common failure modes are goroutine leaks and race conditions.

Goroutine Leaks

A goroutine leak happens when a goroutine blocks forever — typically waiting on a channel that will never receive a value — and is never cleaned up. Because the Go runtime has no automatic timeout for blocked goroutines, a leak quietly accumulates memory until the process is restarted. Consequently, every goroutine you launch should have a clear exit condition, whether that's a closed channel, a cancelled context, or a completed WaitGroup.

Race Conditions and the -race Detector

A race condition occurs when two or more goroutines access the same memory location concurrently, and at least one of them writes to it, without synchronization. Fortunately, Go ships with a built-in race detector — run your tests with go test -race or your binary with go run -race and the tool will flag unsynchronized access at runtime. In particular, it is worth enabling the race detector in continuous integration pipelines rather than relying on it only during local debugging.

Canceling Goroutines with the Context Package

The context package is the idiomatic way to tell a goroutine "stop what you're doing." A context carries a cancellation signal, a deadline, or a timeout down a call chain, and any goroutine that selects on ctx.Done() can exit cleanly the moment that signal fires. This matters enormously in an LLM gateway, where a client disconnecting mid-stream should immediately free the goroutine handling that connection rather than let it keep pulling tokens from an upstream provider nobody is listening to anymore.

The Worker Pool Pattern: Scaling Goroutines in Production

Launching an unbounded number of goroutines for every incoming request can still overwhelm downstream systems, even though each goroutine is cheap. The worker pool pattern solves this by fixing the number of goroutines that process work while allowing the amount of work to vary freely.

1. Create a jobs channel that holds incoming units of work, and a results channel to collect their output once processing finishes.
2. Launch a fixed number of worker goroutines — often tied to GOMAXPROCS — each of which loops over the jobs channel until it closes.
3. Feed jobs into the channel from the main goroutine, closing it once all jobs have been submitted so workers know when to stop.
4. Collect results with a WaitGroup so the main goroutine blocks until every worker has drained the jobs channel and written its output.

This pattern is precisely how River, the PostgreSQL-backed background job library we describe later in this guide, structures its internal job execution — a fixed pool of goroutines pulling work from a queue rather than an unbounded spawn-per-job model.

Real-World Case Study: How Goroutines Cut Memory Use by 75,000x

Everything above is theory until it survives contact with production traffic. What follows is the story of a deliberate migration — not a reaction to outages, but an architectural decision made after building the same backend three times in three different languages. Python added latency to streaming proxy workloads. Ruby's memory model could not sustain thousands of concurrent connections. Both produced container images measured in hundreds of megabytes. The goroutine eliminated all three problems at once. This case study is part of our ongoing From Fork to Framework migration series, where we document every architectural decision in detail.

The Goroutine Advantage in an LLM Gateway

An LLM gateway holds open connections while tokens stream from upstream providers to downstream clients, and each connection lives for seconds to minutes. Under load, a gateway serving a thousand concurrent streaming connections in Rails requires roughly a thousand Puma workers at about 150 megabytes each. The same workload in Go requires a thousand goroutines at roughly 2 megabytes total — a 75,000x reduction in memory consumption for identical concurrency. Because Go's runtime parks blocked goroutines and reschedules runnable ones with no Global Interpreter Lock and no thread-per-request ceiling, this concurrency model was effectively built for exactly this workload pattern.

Benchmarks That Mattered

We did not adopt Go based on microbenchmarks; we adopted it based on benchmarks matching our production workload profile. Bifrost, a Go-based LLM gateway, published results showing 54x lower P99 latency and 9.5x higher throughput than Python equivalents under sustained streaming load. These numbers aligned with what we observed during our own Rails-phase load testing: as concurrent streaming connections increased, Python and Ruby degraded predictably while Go implementations held flat latency curves. For broader context, TechEmpower's Round 23 framework benchmarks place Go frameworks near the top of composite requests-per-second rankings, well ahead of Ruby on Rails, and that gap widens further once connection duration is factored in.

sqlc: Compile-Time SQL Alongside Goroutines

ActiveRecord discovers the database schema at runtime, which is convenient for development but a source of production errors no test suite fully prevents. sqlc takes the opposite approach: you write raw SQL in .sql files, and sqlc reads them alongside your PostgreSQL schema to generate type-safe Go code at compile time. If a query references a nonexistent column, the build fails before it ever reaches a goroutine handling live traffic. Because the PostgreSQL schema from our Rails phase transferred directly — same tables, columns, indexes, and constraints — we simply translated existing queries into .sql files and ran sqlc generate. The pgx driver added native PostgreSQL protocol support, connection pooling, prepared statement caching, and native pgvector support for embedding workloads — all without an ORM abstraction layer sitting between goroutines and the database.

The Production Stack Behind Every Goroutine

chi Router → Handlers → sqlc Generated Code → pgx Connection Pool → PostgreSQL
                      ↘ nhooyr.io/websocket (streaming connections)
                      ↘ River Workers (background jobs, goroutine-based worker pool)
Both services compile to a single static binary (~15 MB) → Distroless container (~18 MB)
                                                            → DigitalOcean App Platform

Both services follow the same pattern. The chi router handles HTTP routing with middleware for authentication, rate limiting, and request logging. Handlers call sqlc-generated functions for database operations, while River — itself a goroutine-powered worker pool — provides PostgreSQL-backed background job processing with no Redis and no external broker. Meanwhile, nhooyr.io/websocket handles the WebSocket connections that stream tokens in real time, each one backed by its own lightweight goroutine.

The entire application compiles to a single static binary, which is copied into a distroless container image containing nothing except the binary and CA certificates — no shell, no package manager, no runtime. If you want to see how a production security platform structures these goroutine-driven services end to end, the AOCore architecture overview walks through the same patterns in detail.

PostgreSQL-Only Infrastructure

One of the most important continuities between the Rails and Go phases is infrastructure topology: both use PostgreSQL as the sole external dependency. Rails achieved this through Solid Queue, Solid Cache, and Solid Cable; Go achieves it through River for job queues, pgx for connection management, and direct PostgreSQL queries everywhere else. We evaluated Redis-backed alternatives at every phase and consistently concluded that a second stateful dependency was not justified for our workload. PostgreSQL's LISTEN/NOTIFY delivers real-time events, and its row-level locking provides job queue semantics, all while the added latency compared to Redis stays in single-digit milliseconds — invisible against LLM response times measured in seconds.

The Migration Strategy

We did not migrate both products simultaneously, and the order was deliberate.

AOSentry came first for three reasons. First, every LLM request flows through it, making it the hottest path in the architecture. Second, its application surface is bounded — primarily a routing and policy engine rather than a feature-rich app. Third, every request to it is a streaming proxy connection, which is exactly where goroutines provide their largest advantage. The AODex migration followed a more methodical process: starting March 21, we wrote Technical Requirements Documents covering ten migration objectives, and on March 24 the first commits landed, beginning the full Rails replacement. This phased approach meant AOSentry was stable in Go before AODex migration began, so lessons about sqlc patterns, middleware design, and River job configuration directly informed the second effort. For teams evaluating how a similar platform documents its investigative tooling around these production decisions, the guide to investigating a guardrail decision is a useful companion resource.

Container Size and Deployment Speed

The difference in container images was not cosmetic. Rails images exceeded 600 megabytes because they bundled Ruby, Bundler, compiled native extensions, precompiled assets, and the full framework. Go images weighed 10 to 20 megabytes because they contained a statically linked binary and nothing else. On DigitalOcean App Platform, image pulls that took 30 to 60 seconds with Rails completed in under 5 seconds with Go, and new instances reached healthy status in milliseconds instead of seconds. For a product handling bursty LLM traffic, that difference closes almost entirely the dangerous window where incoming traffic can exceed current capacity.

What Transferred and What Changed

The PostgreSQL schema transferred completely — every table, column, index, and constraint moved into goose migration files as raw SQL. Business logic transferred conceptually but not syntactically: ActiveRecord callbacks became explicit function calls, Pundit policies became chi middleware, Devise authentication became custom JWT validation, and Solid Queue jobs became goroutine-backed River workers. In every case, the Go version was more verbose and more explicit, and we considered that a feature rather than a cost. What changed most was the error model: Rails rescues exceptions and renders error pages, while Go returns errors as values that must be handled at every call site. This added verbosity, but it eliminated an entire category of bugs where unhandled exceptions produced generic 500 responses in production. New engineers onboarding onto this stack typically start with the AOCore get-started overview before touching any goroutine-heavy code.

Best Practices for Production-Grade Goroutine Usage

• Always give a goroutine an exit path. Whether through a closed channel, a cancelled context, or a WaitGroup, never launch a goroutine that has no defined way to stop.
• Bound concurrency with worker pools instead of spawning one goroutine per request when downstream systems — like a database or a third-party API — can't absorb unlimited parallel load.
• Run tests with the race detector enabled in continuous integration, not just locally, since races often only surface under specific scheduling timing.
• Prefer channels over shared memory for communication between goroutines, reserving sync.Mutex for cases where shared state is genuinely unavoidable.
• Propagate context through every goroutine that performs network I/O so client disconnects and timeouts clean up resources immediately.

Frequently Asked Questions About Goroutines

The Lesson: Why Goroutines Are the Right Primitive

Go's concurrency model is purpose-built for the LLM gateway pattern. When every request is a long-lived streaming connection, a goroutine is not a nice-to-have optimization — it is the right primitive for the problem. That said, this decision was never about Go being universally superior to Rails. Rails built our security platform faster than Go would have, and its conventions accelerated feature development in ways Go's explicit style cannot match. But conventions optimize for developer speed, while goroutines optimize for runtime efficiency, and when your product is a high-throughput streaming proxy that must scale elastically on commodity infrastructure, runtime efficiency is the constraint that matters most. In the end, the goroutine is a small thing. At scale, small things compound — and understanding exactly what a goroutine is, how it's scheduled, and how to synchronize it correctly is the difference between a system that scales gracefully and one that falls over under its own concurrency.