ProgrammingBackend Developer

Explain how memory management works in Python, including the garbage collector. What are cyclic references, and how does Python handle them?

Pass interviews with Hintsage AI assistant

Answer

In Python, memory is managed automatically: when an object is created, memory is allocated, and when no other object references that object, the memory is freed. The main strategy is reference counting: each object has an associated reference count, and when it drops to zero, the object is deleted.

However, cyclic references are possible when objects refer to each other, and their reference count never drops to zero even if the references from external code are removed.

To combat this, Python (CPython) has a garbage collector (GC) that detects cycles of objects that are inaccessible from roots and deletes them. The garbage collector can be managed from the gc module.

Example of a cycle:

class Node: def __init__(self): self.ref = None n1 = Node() n2 = Node() n1.ref = n2 n2.ref = n1 del n1, n2 # objects are still alive due to cyclic reference!

Trick question

Question: "If you call del obj, will the memory always be freed immediately?"

Answer: No! The del operation merely removes one reference to the object. If there are still other references somewhere (either explicitly or cyclically) — memory will not be freed until the reference count drops to zero and/or the GC cleans up the cycles. Example:

import gc class A: pass x = A() y = x del x # y still has a reference, the object is alive! del y # only now can the object be deleted

Examples of real mistakes due to ignorance of the nuances of this topic


Story

In a large file processing system, linked objects were kept in memory. After removing external references, the memory still was not released. The reason was cyclic references that were not immediately cleaned up, and sometimes not cleaned up at all due to the presence of a destructor __del__.


Story

In a long-lived server process, large temporary structures with cyclic graphs were often created. Developers did not turn off temporary GC debug output, leading to rare but noticeable pauses in the background. Diagnosed as "performance drops". It turned out that the frequent triggering of the garbage collector was the culprit.


Story

When porting a large project to PyPy (an alternative interpreter), the old code relied on the specific behavior of CPython for immediate memory clearance through reference counting — in PyPy, objects were deleted and cleared not immediately, but at the "request" of the GC, leading to unpredictable behavior with open files and network connections.