History of the question
Before Python 2.5 introduced the with statement via PEP 343, resource management required explicit try/finally blocks scattered throughout codebases. While functional, this pattern was verbose and error-prone for simple resource acquisition and release scenarios. The contextlib module was introduced to reduce this boilerplate by allowing developers to write context managers as generator functions, using the @contextmanager decorator to transform sequential-looking generators into objects satisfying the context management protocol.
The problem
A generator function natively implements the iterator protocol (__iter__, __next__), not the context manager protocol (__enter__, __exit__). The fundamental challenge lies in bridging these distinct protocols: when entering a with block, the setup code before the yield must execute; when exiting, the cleanup code after the yield must run regardless of exceptions. Furthermore, exceptions raised inside the with block must be injectable back into the generator at the exact yield suspension point, allowing the generator's own exception handling logic to execute cleanup operations.
The solution
The decorator wraps the generator function in a GeneratorContextManager class (implemented in C in modern CPython). Each invocation creates a fresh generator iterator. The __enter__ method calls next() on this iterator, executing the function until the yield statement, and returns the yielded value to be bound to the as variable. The __exit__ method receives exception details; if no exception occurred, it calls next() again to resume and exhaust the generator. If an exception occurred, it calls the generator's throw() method, injecting the exception at the suspended yield point. This allows the generator's except or finally blocks to handle cleanup. If throw() returns normally (exception caught), __exit__ returns True to suppress the exception; otherwise, it propagates.
from contextlib import contextmanager @contextmanager def managed_connection(): conn = create_connection() try: print("Connection established") yield conn except NetworkError: conn.rollback() raise finally: conn.close() print("Connection closed") with managed_connection() as c: c.query("SELECT * FROM data")
Problem description: A high-throughput data processing service needed to handle temporary spill files when in-memory buffers exceeded limits. The legacy implementation duplicated file creation and deletion logic across 12 different processing modules, leading to file descriptor leaks during edge-case error conditions and complicating maintenance.
Solutions considered:
Manual try/finally blocks were the initial approach. Every usage site wrapped file operations in explicit try/finally to ensure os.unlink() was called. This offered explicit control flow with zero abstraction overhead, but proved verbose at eight lines per usage site and highly error-prone. Developers occasionally placed cleanup logic in the wrong finally block, and modifying behavior consistently across all modules was arduous when logging requirements were added.
A class-based context manager was considered as a reusable alternative. A TempSpillFile class would implement __enter__ to create the file and __exit__ to delete it. While reusable and following the standard protocol, the class definition visually separated setup from cleanup by many lines, harming readability. It also required fifteen lines of boilerplate for what was conceptually a simple resource lifecycle, obscuring the actual logic.
The generator with @contextmanager approach was the final option. A temp_spill_file() generator function would create the file, yield it, and use try/finally for deletion. This minimized code duplication and kept setup and cleanup adjacent in the source code, leveraging familiar exception handling syntax. However, it imposed a single-use limitation and the yield suspension point could confuse developers expecting synchronous execution.
Chosen solution and result: The @contextmanager approach was selected because it minimized code duplication while maximizing clarity during code reviews. The adjacency of acquisition and release logic made the resource lifecycle immediately obvious. The refactoring reduced the resource management code from ninety-six lines to twelve lines across the codebase. Static analysis confirmed zero file descriptor leaks during the subsequent quarter of production use.
How does GeneratorContextManager handle exceptions that occur during the setup phase (before the yield) versus the cleanup phase (after the yield)?
If an exception occurs before the yield in the generator, the generator never suspends; __enter__ propagates this exception immediately and __exit__ is never invoked. If an exception occurs within the with block (after yield), the generator is suspended. __exit__ then calls generator.throw(exc_type, exc_val, exc_tb), which resumes the generator at the yield line with the exception active. This allows the generator's own except or finally blocks to execute. Candidates often miss that throw() actually resumes execution and that the exception is considered to occur at the yield expression from the generator's perspective.
Why does a contextmanager-decorated generator enforce a single yield point, and what specific error occurs if this constraint is violated?
The context manager protocol assumes a single entry and exit. If the generator yields a second time—either because __exit__ calls next() (no exception) and the generator yields again instead of returning, or because throw() is called and the generator handles the exception then yields again—the GeneratorContextManager raises RuntimeError with the message "generator didn't stop". This occurs because the state machine expects the generator to be exhausted after cleanup. Candidates frequently confuse this with standard iteration where multiple yields are valid, not realizing the yield acts as a suspend/resume boundary for the context, not a value production sequence.
Under what circumstances does the __exit__ method of a GeneratorContextManager suppress an exception raised in the with block, and how does this interact with the generator's exception handling?
__exit__ suppresses the exception (returns True) only if the injected exception via throw() is caught within the generator and the generator reaches its end (raises StopIteration) without re-raising the exception or raising a new one. If the generator catches the exception and allows the throw() call to return normally, __exit__ interprets this as successful handling and returns True. If the generator does not catch the exception, throw() propagates it out, and __exit__ returns None (falsy), allowing the exception to propagate. Candidates often miss that simply having a try/except inside the generator isn't enough; the exception must be caught specifically from the throw() call and not re-raised, and that an explicit return or falling off the end after catching is required for suppression.