Python Generators
In this page:
def generator_name():
yield value1
yield value2
for item in generator_name():
# use itemWhat is a Generator?
A generator function looks like an ordinary function but uses yield instead of return, and calling it doesn't run the function body immediately -- it returns a generator object that produces values one at a time only as they're requested.
Example: What is a Generator?
def my_gen():
yield 1 # yield instead of return makes this a generator function
yield 2
g = my_gen()
print(type(g)) # generator object, body hasn't run yet
print(next(g)) # runs up to the first yieldThe yield Keyword
Each time yield executes, the function pauses exactly there, hands back the yielded value, and remembers its entire local state (variables, position in loops) -- the next call to next() resumes execution right after that yield rather than starting the function over.
Example: The yield Keyword
def counter():
n = 1
while True:
yield n # pauses here, remembering n
n += 1
c = counter()
print(next(c)) # first call, prints 1
print(next(c)) # second call, resumes after yield and prints 2Generator Expressions
A generator expression -- (x*2 for x in range(10)) -- creates a generator with syntax almost identical to a list comprehension, but using parentheses instead of square brackets, and without ever building the full list of results in memory.
Example: Generator Expressions
squares = (x * x for x in range(5))
print(list(squares))Generator State and Lifecycle
When a generator function runs off the end of its body or hits a plain return statement, Python automatically raises StopIteration to signal there's nothing left to produce -- this is exactly the same signal a for loop watches for when iterating any iterator.
Example: Generator State and Lifecycle
def one_value():
yield "only value"
g = one_value()
print(next(g))
try:
next(g) # generator body has finished
except StopIteration:
print("Generator exhausted")Performance Benefits
Because a generator computes and yields one value at a time instead of building an entire list upfront, it uses a fixed, small amount of memory regardless of how many values it ultimately produces -- ideal for streaming through datasets too large to fit comfortably in RAM.
Example: Performance Benefits
import sys
list_version = [x for x in range(1000)] # builds the entire list upfront
gen_version = (x for x in range(1000)) # produces values lazily, one at a time
print(sys.getsizeof(list_version) > sys.getsizeof(gen_version))- Looping over a generator twice and getting nothing the second time, because it is exhausted after one pass.
- Using
return valuewhere you meantyield value, so the function runs once instead of producing a sequence. - Converting a large generator to a list with
list(gen), which loads everything into memory and loses the benefit.
Chapter Quiz — Complete all 12 topics to unlock
0/12 topics done
Complete these topics first: