PythonProgrammingSenior Python Developer

How does the presence of `__set__` in a **Python** descriptor alter the precedence of instance dictionaries during attribute resolution?

Pass interviews with Hintsage AI assistant

Answer to the question

History of the question

In early Python versions, attribute resolution relied on a simple depth-first search through the instance dictionary followed by the class hierarchy. This approach proved insufficient for implementing robust property-like behavior where computed values needed to intercept both reads and writes without ambiguity. The introduction of new-style classes in Python 2.2 established the descriptor protocol, categorizing descriptors based on the presence of __set__ or __delete__ to solve precedence conflicts.

The problem

Without a strict precedence rule, the interpreter could not decide whether an instance's local storage should override class-level definitions or vice versa. If instance dictionaries always took precedence, properties could not validate assignments because values would be stored directly in __dict__. Conversely, if class attributes always dominated, normal instance variables would be inaccessible when names collided with methods or other class attributes.

The solution

Python's attribute lookup algorithm mandates that data descriptors—those defining __set__ or __delete__—take precedence over instance dictionaries, while non-data descriptors (defining only __get__) yield to instance dictionaries. This design allows @property to enforce validation logic by intercepting writes, while ordinary functions or cached properties remain overridable per instance without complex metaprogramming.

Situation from life

A development team was building a high-throughput data validation layer for a financial trading platform. They required persistent fields that strictly validated incoming market data against regulatory constraints, ensuring no invalid values could be assigned. Additionally, they needed computed metrics that could be cached per instance to avoid expensive recalculation of volatility indices during high-frequency trading bursts.

Solution 1: Universal properties

One approach considered was implementing all attributes as properties using the @property decorator. This provided comprehensive validation control by intercepting every write operation through the property's setter method. However, this design prevented the system from bypassing validation when loading serialized data from trusted internal caches, creating unnecessary computational overhead during bulk replay operations.

Solution 2: Centralized setattr

Another option involved overriding __setattr__ on the base class to centralize validation logic within a single method. While this centralized control offered a single point of modification for validation rules, it introduced fragile branching logic to distinguish between persistent fields requiring validation and temporary computational caches. Furthermore, this approach interfered with standard attribute access patterns expected by third-party serialization libraries, causing integration failures.

Chosen solution

The chosen solution leveraged the descriptor protocol's dichotomy directly to satisfy both requirements without centralization overhead. The team implemented ValidatedField as a data descriptor with a __set__ method enforcing type and range constraints, ensuring it always intercepted assignments regardless of instance state because data descriptors take precedence over instance dictionaries. For computed metrics, they created CachedMetric as a non-data descriptor implementing only __get__, allowing the instance dictionary to shadow the descriptor once a value was computed and stored locally, thereby bypassing recalculation on subsequent accesses.

Result

This architecture provided strict validation for external inputs while permitting flexible, performant caching for derived values. The system successfully processed high-volume market feeds without validation bottlenecks during cache hydration. Benchmarking revealed a 40% reduction in validation overhead during historical replay scenarios compared to the property-only approach, while maintaining full regulatory compliance for live data ingestion.

What candidates often miss

Does deleting an attribute bypass a data descriptor if the descriptor lacks a __delete__ method?

When a data descriptor implements __set__ but omits __delete__, attempting to delete the attribute via del obj.attr does not fall back to the instance dictionary. Python still recognizes the object as a data descriptor due to the presence of __set__, and the deletion operation will raise an AttributeError indicating the attribute cannot be deleted. To allow deletion, the descriptor must explicitly define __delete__ to remove the value from the instance, or the class must implement custom deletion logic; the lookup mechanism never checks the instance dictionary for data descriptor attributes during deletion operations.

Why does super().attribute appear to ignore data descriptors defined on the current class?

The super() proxy implements a cooperative multiple inheritance mechanism that begins searching the Method Resolution Order (MRO) at the class following the current class in the hierarchy. Since the descriptor is defined on the current class itself, super() skips it during lookup. However, if a parent class defines a data descriptor with the same name, super() will find it and apply the standard data descriptor precedence rules, invoking __get__ with the instance and owner class appropriately. This behavior stems from the MRO starting point, not from any special exemption for descriptors in super proxy objects.

How do __slots__ utilize the descriptor protocol to enforce storage constraints?

When a class defines __slots__, the Python interpreter automatically creates specialized internal descriptors (typically member_descriptor objects at the C level) for each slot name and places them in the class dictionary. These descriptors implement both __get__ and __set__, making them data descriptors that take precedence over any attempt to store values in a conventional instance dictionary. Since instances of slotted classes typically lack a __dict__ unless "__dict__" is explicitly included in the slots list, the descriptor protocol ensures all reads and writes for slotted attributes are channeled through these C-level descriptors, enforcing type safety and memory efficiency by preventing arbitrary attribute attachment.