The C10K problem challenged early 2000s server architectures to handle ten thousand concurrent connections efficiently. Traditional one-thread-per-connection models exhausted memory and CPU through context switches. Go’s creators aimed to support millions of goroutines while preserving the clarity of blocking I/O code, necessitating a mechanism to decouple goroutine waiting from OS thread consumption.
When a goroutine executes a blocking system call—such as read() on a network socket—it risks pinning the underlying OS thread (M). Without intervention, thousands of concurrent connections would spawn thousands of threads, negating the M:N scheduling advantages and exhausting system resources.
The Go runtime employs a network poller (utilizing epoll on Linux, kqueue on BSD, and IOCP on Windows) integrated directly into the scheduler. When a goroutine initiates I/O on a pollable descriptor, the runtime parks it in _Gwaiting state and registers the file descriptor with the OS-specific poller. A monitoring thread waits for readiness; upon notification, the poller transitions the goroutine to _Grunnable and schedules it onto an available P (logical processor). This transforms blocking operations into efficient parking events, allowing a small GOMAXPROCS thread pool to service massive concurrency.
// Idiomatic Go code that actually parks rather than blocks func handleConn(conn net.Conn) { buf := make([]byte, 1024) n, err := conn.Read(buf) // Parks goroutine, frees thread if err != nil { log.Println(err) return } process(buf[:n]) }
You are building a high-frequency trading gateway that maintains 20,000 persistent TCP connections to market data feeds. During volatility spikes, latency must remain below 100 microseconds. Initial testing using a Java NIO approach achieved throughput but suffered complex callback maintenance. When migrating to Go, the team wrote straightforward blocking code using net.TCPConn. However, under load testing with 50k concurrent connections, the process spawned over 10,000 OS threads, triggering OOM kills and destroying latency guarantees.
Solution A: Reimplement reactor pattern manually. Bypass the standard library and use syscall wrappers to create a manual epoll event loop with buffer pooling. Pros: Maximum control over memory layout and wake-up latency. Cons: Sacrifices Go’s sequential coding model, introduces platform-specific complexity, and duplicates battle-tested runtime code, increasing bug surface area.
Solution B: Accept thread overhead with runtime.LockOSThread. Force each connection onto a dedicated thread to guarantee scheduling isolation. Pros: Predictable thread affinity. Cons: Violates the fundamental economic benefit of goroutines; memory usage balloons to ~8MB per connection, rendering the approach infeasible for the target scale.
Solution C: Audit for non-pollable I/O and trust the netpoller. Retain idiomatic blocking code but eliminate accidental blocking syscalls (e.g., file logging or DNS lookups without resolver awareness) that force thread creation. Pros: Maintains readable linear flow; leverages runtime optimizations across Linux/macOS/Windows; reduces memory to ~2KB per connection. Cons: Requires deep understanding that net.Conn operations park while os.File operations block threads.
The team selected Solution C, recognizing that the thread explosion stemmed from logging market data to local ext4 files synchronously within the hot path. Regular file I/O cannot use the netpoller (files are always "ready" in Unix epoll), so each log write blocked an OS thread. They refactored to use an async file writer goroutine with a channel buffer, keeping network I/O (which is pollable) on the main goroutines.
The gateway now sustains 50,000 connections with only 16 OS threads (matching GOMAXPROCS), achieving ~85µs P99 latency. Memory consumption dropped from 40GB (projected thread stacks) to ~180MB total RSS.
Why does reading from os.Stdin or a regular file block an OS thread despite using the same Read method as a TCP socket, and how does this affect CLI tool concurrency?
While TCP sockets support asynchronous readiness notifications via epoll, regular files and pipes on Unix systems always report as "ready" for I/O; the kernel provides no non-blocking interface for file data availability. Consequently, when a goroutine calls os.File.Read, the Go runtime cannot park it—it must dedicate a real OS thread to the blocking syscall. In CLI tools that spawn goroutines per input file (e.g., log processors), this causes thread leakage identical to traditional threading models. The solution limits concurrent file operations using semaphores or uses buffering with dedicated worker pools.
How does the runtime prevent a "thundering herd" when the netpoller simultaneously wakes thousands of goroutines after a network partition heals?
When the netpoller (via epoll_wait) returns thousands of ready descriptors, the netpoll function distributes goroutines across all Ps (logical processors) using the global run queue and work-stealing algorithms, rather than enqueueing them all onto a single P. Additionally, the scheduler implements fairness ticks: after every 10ms of execution, it checks for runnable I/O goroutines to prevent CPU-bound tasks from starving them. Candidates often assume FIFO queuing per connection, missing that the scheduler balances throughput by spreading wake-up events and enforcing preemption points.
What race condition exists between SetReadDeadline and an active Read call, and why does the timer wheel implementation require atomic synchronization with the netpoller?
The netpoller uses a per-P timer wheel or min-heap to manage I/O deadlines. When goroutine A calls SetReadDeadline while goroutine B blocks in Read, A modifies the timer that B’s parked state depends upon. Without atomic updates (protected by internal mutexes in net.conn), a race could occur where the poller observes the old deadline after the new one is set, causing a missed wakeup (indefinite hang) or a spurious timeout. The atomicity ensures happens-before consistency: either the updated deadline is observed by the epoll wait cycle, or the previous timer fires, but never an undefined intermediate state that violates the deadline contract.