Answer to the question
Python implements lexical scoping via a mechanism involving cell objects that act as intermediaries between nested functions and their enclosing scopes. When a nested function references a variable from an outer scope, the compiler marks it as a free variable (stored in co_freevars) and the enclosing function stores that variable's value inside a cell object rather than a standard local variable slot. The nonlocal keyword instructs the interpreter to resolve the name lookup to this existing cell object rather than creating a new local binding, thereby allowing the inner scope to read and write to the same memory location as the outer scope.
Situation from life
We needed to implement a lightweight audit logger for a data processing pipeline that would maintain a running count of sanitized records across multiple callback invocations without polluting the global namespace or creating a full class hierarchy. The challenge was ensuring that the counter state persisted between calls to the inner logging function while remaining encapsulated within the factory function that created it.
One solution considered was using a global dictionary to store counters keyed by logger ID. This approach offered simplicity and allowed external inspection of state, but introduced global namespace pollution and required complex locking mechanisms to ensure thread safety across the entire application. Additionally, it broke encapsulation by exposing implementation details to other modules.
Another approach involved creating a dedicated class with an instance attribute to hold the counter. This provided proper encapsulation and familiar object-oriented semantics, but added unnecessary boilerplate for what was essentially a single-function utility, and the overhead of instance creation was deemed excessive for a high-frequency logging operation that would be instantiated thousands of times.
The chosen solution utilized a closure with the nonlocal declaration to bind the counter to a cell object in the enclosing scope. This approach maintained clean functional encapsulation without class overhead, ensured the state remained private to the closure, and leveraged Python's optimized cell dereferencing mechanism which, while slightly slower than local variables, was negligible compared to I/O operations. The result was a 40% reduction in memory overhead compared to the class-based approach and elimination of global state conflicts.
What candidates often miss
Why does assignment to a variable from an outer scope create a new local variable instead of modifying the outer one without the nonlocal keyword?
In Python, assignment is a statement that binds a name to a value within the current local scope by default. When the compiler encounters an assignment inside a nested function, it determines that the variable is local to that function unless declared otherwise. Without nonlocal, the inner function creates a new entry in its own f_locals dictionary, completely shadowing the outer variable. The nonlocal declaration forces the compiler to treat the variable as a reference to the cell object created in the enclosing scope, allowing read and write access to the shared memory location.
What is the fundamental difference between nonlocal and global regarding scope resolution?
While both keywords modify the scope in which an assignment operates, global restricts name resolution to the module-level global namespace, bypassing any intervening enclosing function scopes. In contrast, nonlocal specifically skips the current local scope and searches through enclosing function definitions (but not the module globals) to find the nearest cell object associated with the name. This means nonlocal cannot be used to modify module-level variables, and global cannot see variables inside nested functions unless they are explicitly declared global in those outer functions as well.
How do multiple nested functions share the same state via cell objects, and when are these cells actually allocated?
When an outer function defines multiple inner functions that reference the same variable from the outer scope, the Python compiler creates a single cell object for that variable in the outer function's frame. All inner functions receive a reference to this same cell object in their __closure__ tuple. These cells are allocated at runtime when the outer function executes (not when the code is compiled), and they persist as long as any inner function (or reference to them) exists. This shared cell object is what enables the different inner functions to observe each other's modifications to the enclosed variable, creating a shared state mechanism similar to instance variables but without classes.