Every for loop in Python is really the iterator protocol at work: iter() to get an iterator, then next() repeatedly until it signals it is finished. Lists, files, dicts and strings all speak it, which is why one loop shape works on all of them.
Iterators are the objects behind Python’s for loops. We can manually step through them using iter() and next():
my_list = [1, 2, 3]
my_iter = iter(my_list)
print(next(my_iter))
print(next(my_iter))
print(next(my_iter))1 2 3
Because it is a protocol rather than a base class, your own classes can join in: implement __iter__ and __next__ and your object works in a for loop like anything built in.
When an iterator is exhausted, calling next() raises a StopIteration exception:
my_list = [10]
my_iter = iter(my_list)
print(next(my_iter))
try:
print(next(my_iter))
except StopIteration:
print("Caught StopIteration!")10 Caught StopIteration!
Writing a class that implements the iterator protocol can be tedious. Generators simplify this using the yield keyword.
def count_to_three():
yield 1
yield 2
yield 3
gen = count_to_three()
print(next(gen))
print(next(gen))
print(next(gen))1 2 3
A generator does the same job with far less ceremony. Each yield hands back a value and freezes the function — local variables, position, everything — so the next next() resumes exactly where it stopped.
Here is an IP01 interactive visualizer that shows how the generator pauses at each yield and resumes exactly where it left off, saving its state:
def stateful_generator():
print("Starting")
yield 1
print("Resuming")
yield 2
gen = stateful_generator()
print("Before first next()")
print(next(gen))
print("Before second next()")
print(next(gen))For one-liners there is the generator expression: the same syntax as a list comprehension, but with parentheses. It produces values on demand instead of building the whole list up front.
Instead of a full function, you can create quick generators using generator expressions:
squares_gen = (x * x for x in range(4))
for sq in squares_gen:
print(sq)0 1 4 9
Complete the interactive challenges below to earn XP!
Is the following statement true or false? 'Generators load all their values into memory at once like a list.'
False — Generators use lazy evaluation, producing values one at a time on demand, making them highly memory efficient for large datasets.
Is the following statement true or false? 'You can reuse a generator multiple times.'
False — Generators are exhausted after they are iterated through once. To iterate again, you must create a new generator object.
Is the following statement true or false? 'A generator function returns a value immediately when called.'
False — Calling a generator function returns a generator object. It does not begin execution until next() is called.
Write a generator function `even_numbers(max_val)` that yields even numbers from 0 up to `max_val` (inclusive).
Use a for loop and the modulo operator % to check if a number is even, then yield it.
def even_numbers(max_val):
for i in range(max_val + 1):
if i % 2 == 0:
yield i
for num in even_numbers(4):
print(num)Use a generator expression to create an iterator named `cubes_gen` that yields the cube of numbers from 0 to 3.
The syntax for a generator expression uses parentheses: (expression for item in iterable).
cubes_gen = (x**3 for x in range(4))
for cube in cubes_gen:
print(cube)