Skip to main content Brad's PyNotes

PEP 525 - Asynchronous Generators

TL;DR

PEP 525 introduced asynchronous generators to Python 3.6, enabling functions that combine async def with yield statements. This feature simplifies creating asynchronous data sources by replacing verbose iterator classes with concise generator syntax, delivering approximately 2.3x better performance.

Interesting!

The asend() method enables bidirectional communication with asynchronous generators, allowing you to push values into them just like regular generators’ send() method, but in an async context.

The Problem Before PEP 525

Before asynchronous generators existed, creating an asynchronous data source required implementing a full class with __aiter__() and __anext__() methods:

python code snippet start

class Ticker:
    def __init__(self, delay, to):
        self.delay = delay
        self.i = 0
        self.to = to

    def __aiter__(self):
        return self

    async def __anext__(self):
        i = self.i
        if i >= self.to:
            raise StopAsyncIteration
        self.i += 1
        if i:
            await asyncio.sleep(self.delay)
        return i

python code snippet end

This verbose boilerplate was necessary for even simple asynchronous iteration patterns.

The Elegant Solution

Asynchronous generators make this dramatically simpler by combining async def with yield:

python code snippet start

async def ticker(delay, to):
    """Yield numbers from 0 to `to` every `delay` seconds."""
    for i in range(to):
        yield i
        await asyncio.sleep(delay)

python code snippet end

This concise syntax mirrors how regular generators simplified synchronous iteration.

Using Asynchronous Generators

Asynchronous generators integrate seamlessly with async for loops:

python code snippet start

async def main():
    async for value in ticker(1, 5):
        print(value)

# Output (with 1 second delays):
# 0
# 1
# 2
# 3
# 4

python code snippet end

The generator automatically handles the asynchronous iteration protocol.

Generator Methods

Asynchronous generators provide four key methods:

__anext__() - Returns an awaitable for the next value:

python code snippet start

gen = ticker(1, 3)
print(await gen.__anext__())  # 0
print(await gen.__anext__())  # 1

python code snippet end

asend(val) - Pushes values into the generator:

python code snippet start

async def echo():
    while True:
        value = yield
        print(f"Received: {value}")

gen = echo()
await gen.asend(None)  # Prime the generator
await gen.asend("hello")  # Received: hello

python code snippet end

athrow(typ, val, tb) - Throws exceptions into the generator:

python code snippet start

async def resilient():
    count = 0
    try:
        while True:
            count += 1
            await asyncio.sleep(0.1)
            yield f"item-{count}"
    except ValueError:
        print("Caught ValueError, continuing")
        yield "recovered"

gen = resilient()
print(await gen.__anext__())  # "item-1"
print(await gen.athrow(ValueError))  # Caught ValueError, continuing / "recovered"

python code snippet end

aclose() - Safely closes the generator and executes cleanup:

python code snippet start

async def managed_resource():
    print("Opening resource")
    await asyncio.sleep(0.1)
    try:
        for i in range(10):
            yield i
            await asyncio.sleep(0.1)
    finally:
        print("Closing resource")  # Always executes
        await asyncio.sleep(0.1)

gen = managed_resource()
print(await gen.__anext__())  # Opening resource / 0
print(await gen.__anext__())  # 1
await gen.aclose()  # Closing resource (cleanup runs even though iteration incomplete)

python code snippet end

Safe Finalization

A critical feature of PEP 525 is ensuring asynchronous generators clean up properly when iteration stops early:

python code snippet start

async def file_processor():
    print("Opening file connection")
    file_handle = "file.txt"  # Simulated resource
    try:
        for i in range(1000):
            await asyncio.sleep(0.01)
            yield i ** 2
    finally:
        print(f"Closing {file_handle}")  # Cleanup always runs
        await asyncio.sleep(0.01)

# Break early - cleanup still executes
async for i in file_processor():
    if i == 100:  # i is 10^2
        break  # "Closing file.txt" still prints

python code snippet end

Event loops use sys.set_asyncgen_hooks() to intercept generator finalization, ensuring cleanup code executes even when generators are abandoned mid-iteration.

Restrictions for Safety

To prevent resource leaks, asynchronous generators cannot yield inside finally blocks:

python code snippet start

async def unsafe():
    try:
        yield 1
    finally:
        yield 2  # Not allowed!

async def test():
    gen = unsafe()
    print(await gen.__anext__())  # 1
    await gen.aclose()  # RuntimeError: asynchronous generator ignored GeneratorExit

asyncio.run(test())

python code snippet end

This restriction ensures cleanup code completes without suspension.

Performance Benefits

The PEP’s implementation benchmarks showed asynchronous generators are approximately 2.3x faster than equivalent class-based async iterators, making them both more convenient and more efficient.

PEP 525 extended Python’s asynchronous programming capabilities by bringing generator elegance to async contexts. Combined with async/await syntax, asynchronous generators provide powerful tools for building responsive, efficient asynchronous data pipelines.

async/await syntax established the foundation that made asynchronous generators possible, while the asyncio module provides the event loop infrastructure that manages generator finalization.

Reference: PEP 525 - Asynchronous Generators