Python Generators: Efficient Iteration and Advanced Use Cases

Python Generators

Discover how Python generators simplify iteration, reduce boilerplate, and improve memory usage compared to traditional iterators.

What Are Generators?

Building a custom iterator in Python usually requires a class that implements __iter__() and __next__(), manages internal state, and raises StopIteration when finished. Generators eliminate this boilerplate by allowing you to write a simple function that yields values one at a time.

In short, a generator is a function that returns an iterator. The function’s body executes lazily, pausing after each yield and resuming on subsequent calls.

Create Generators in Python

Defining a generator is as simple as writing a normal function and replacing a return with yield:

def my_gen():
    n = 1
    print('This is printed first')
    yield n
    n += 1
    print('This is printed second')
    yield n
    n += 1
    print('This is printed at last')
    yield n

Calling my_gen() returns an iterator that starts executing only when next() is called.

a = my_gen()
next(a)  # prints first message and returns 1
next(a)  # prints second message and returns 2
next(a)  # prints last message and returns 3
next(a)  # raises StopIteration

Local variables persist between yields, but the generator can be iterated only once. To start over, create a new instance.

Because a generator is an iterator, you can loop over it directly:

for item in my_gen():
    print(item)

Common Generator Patterns

Reversing a String

def rev_str(s):
    for i in range(len(s)-1, -1, -1):
        yield s[i]

for char in rev_str('hello'):
    print(char)

Works with any iterable (list, tuple, etc.).

Generator Expressions

Anonymous generators can be created with parentheses, similar to list comprehensions:

my_list = [1, 3, 6, 10]
# List comprehension
squares_list = [x**2 for x in my_list]
# Generator expression
squares_gen = (x**2 for x in my_list)
print(squares_list)
print(squares_gen)

The generator yields values on demand:

a = (x**2 for x in my_list)
print(next(a))  # 1
print(next(a))  # 9
print(next(a))  # 36
print(next(a))  # 100
next(a)  # StopIteration

Generator expressions can be passed directly to functions like sum or max without the surrounding parentheses.

Why Use Generators?

• Concise Implementation: Replaces verbose iterator classes with a few lines of code.
• Memory Efficiency: Produces one item at a time, ideal for large or infinite sequences.
• Infinite Streams: Represent endless data sources (e.g., while True: yield).
• Pipeline Composition: Chain multiple generators for clean, readable data processing.

Example: Power of Two

def pow_two_gen(max_n=0):
    n = 0
    while n < max_n:
        yield 2 ** n
        n += 1

Example: Summing Squares of Fibonacci Numbers

def fibonacci_numbers(count):
    a, b = 0, 1
    for _ in range(count):
        a, b = b, a + b
        yield a

def square(nums):
    for n in nums:
        yield n ** 2

print(sum(square(fibonacci_numbers(10))))  # 4895

Generators enable elegant, efficient, and maintainable code for a wide range of iteration scenarios.