Answer.

The for-in loop in Python is implemented through the iteration protocol, which is based on two special methods: __iter__() and __next__(). When the loop begins, Python calls iter(obj) (it looks for the __iter__ method). If it returns an iterator object where __next__() is defined, the loop starts calling this method to get each subsequent element.

If the element is successfully returned, iteration continues. When a StopIteration exception is raised, the loop ends.

Example of a custom iterator:

class Counter:
    def __init__(self, low, high):
        self.current = low
        self.high = high
    def __iter__(self):
        return self
    def __next__(self):
        if self.current > self.high:
            raise StopIteration
        else:
            self.current += 1
            return self.current - 1

for num in Counter(1, 3):
    print(num)  # Outputs 1, 2, 3

Subtlety: Many standard Python structures implement the iterator protocol; you can easily make your objects the same.

Trick question.

Can you use the for-in loop with any Python object? How is an iterable object different from an iterator?

Answer: No, the for-in loop requires the object to be iterable, i.e., to implement the __iter__() method which returns an iterator (an object with the __next__() method). The iterator returns itself through __iter__(), while the iterable object does not.

Example:

lst = [1, 2, 3]
iterator = iter(lst)
print(hasattr(lst, '__iter__'))         # True
print(hasattr(iterator, '__next__'))    # True (iterator)
print(hasattr(lst, '__next__'))         # False (not an iterator)

Examples of real errors due to lack of knowledge of the nuances of the topic.

---

History

Project: Processing large files.

Problem: In the project, they tried to iterate over a file object (iterate over the lines of the file) twice without explicitly opening a new file — and on the second iteration did not receive any lines (the pointer was already at the end of the file!).

---

History

Project: Integrating their own collection class with the framework.

Problem: The class implemented only __getitem__ but not __iter__, and was not compatible with generators and standard loops (for). As a result, third-party libraries that expected an iterator broke down.

---

History

Project: Using map/filter on custom collections.

Problem: The custom list class did not implement __iter__ and __next__, and turned out to be incompatible with many standard Python functions, which only became apparent after integration tests and required significant redesign.