How are iterators and generators implemented and work in Perl? What common patterns of lazy evaluation are used, and what are their limitations?

Answer

In Perl, there are no native generators like in Python, but lazy evaluation and iterators can be implemented using closures, state tracking in lexical variables, and generator functions:

sub counter {
    my $x = shift;
    return sub { return $x++; };
}

my $it = counter(5);
print $it->(), ", ", $it->(); # 5, 6

For complex iterators, CPAN modules (Iterator::Simple, List::Gen) are often used. The classic lazy pattern is returning an anonymous subroutine with preserved state.

Cons: there is no built-in support for yield, many CPAN modules lack composability. Recursion is also limited by stack sizes.

Trick Question

Can an infinite lazy list of Fibonacci numbers be implemented in Perl without running out of memory?

Answer: Yes, using a closure:

sub fibonacci {
    my ($a, $b) = (0, 1);
    return sub { ($a, $b) = ($b, $a+$b); return $a; };
}
my $fib = fibonacci();
print $fib->(), ", ", $fib->(), ", ", $fib->();

But if you store results in an array, it will eventually run out of memory (i.e., only the generator itself is truly "lazy").

Examples of real errors due to ignorance of the nuances of the topic

---

History

A custom iterator for traversing a huge file was written in a project, implemented via an array inside an object. The iterator loaded the entire file into memory — and as the file grew, the service started to experience OOM when dealing with multiple instances.

---

History

A closure-generator for a sequence of details for a report led to an unexpected memory leak — inside the closure, a reference to a large input data array was inadvertently retained, preventing the garbage collector from functioning.

---

History

An attempt to implement a complex generator using recursion without tracking depth led to exceeding the stack limit when processing truly large data, instead of the expected "lazy" traversal.