Async Programming Fundamentals

Master asyncio, coroutines, and event loops in Python.

What is Asynchronous Programming?

Asynchronous programming allows your program to handle multiple operations concurrently without blocking. Instead of waiting for one task to complete before starting another, async code can start multiple tasks and switch between them while waiting for I/O operations (network requests, file operations, database queries) to complete.

Why Use Async Programming?

  • I/O-bound operations: Handle thousands of concurrent connections
  • Better resource utilization: Don't waste CPU while waiting
  • Responsive applications: UI stays responsive during operations
  • Scalability: Serve more users with same hardware
  • Modern web development: Essential for APIs, websockets, microservices

Synchronous vs Asynchronous Execution

⏸️ Synchronous (Blocking)
Task 1: Start → Wait → Complete
⬇️
Task 2: Start → Wait → Complete
⬇️
Task 3: Start → Wait → Complete

Total time: Sum of all tasks
Problem: CPU idle while waiting

⚡ Asynchronous (Non-blocking)
Task 1: Start → ⏳
Task 2: Start → ⏳
Task 3: Start → ⏳
⬇️ (all running concurrently)
✓ All Complete

Total time: Longest single task
Benefit: Maximum efficiency

The Event Loop: Heart of Async Python

The event loop is the core of asyncio. It manages and executes asynchronous tasks, switching between them when they're waiting for I/O operations.

How the Event Loop Works:
  1. Schedule tasks: Add coroutines to the event loop
  2. Run until await: Execute code until it hits an await
  3. Suspend & switch: Pause current task, switch to another ready task
  4. Resume when ready: Continue paused tasks when I/O completes
  5. Repeat: Continue until all tasks complete
import asyncio

async def task1():
    await asyncio.sleep(1)
    print("task1 done")

async def task2():
    await asyncio.sleep(1)
    print("task2 done")

async def task3():
    await asyncio.sleep(1)
    print("task3 done")

async def main():
    print("Event loop started")
    # Tasks scheduled here run concurrently
    await asyncio.gather(
        task1(),
        task2(),
        task3()
    )
    print("Event loop finishing")

asyncio.run(main())
Expected Output:
Event loop started
task1 done
task2 done
task3 done
Event loop finishing

asyncio.run() is the entry point to use from Python 3.7 onwards: it creates the event loop, runs main() to completion, and shuts the loop down afterwards. The three tasks each sleep for a second, but they sleep at the same time, so the whole program finishes in about one second rather than three. That overlap is the entire point of the loop, and it is the one thing the printed output above does not show you.

Getting the loop object: when you need the loop itself, ask for it from inside a coroutine with asyncio.get_running_loop(). Reach for the older asyncio.get_event_loop() at module level and Python 3.12 answers with DeprecationWarning: There is no current event loop, because at that point there is no loop running to hand back.
import asyncio

async def main():
    # Ask for the loop from inside a coroutine, where one is running
    loop = asyncio.get_running_loop()
    print(type(loop).__name__)

asyncio.run(main())
Expected Output:
_UnixSelectorEventLoop

Coroutines: The Building Blocks

Coroutines are special functions defined with async def. They can be paused and resumed, allowing other code to run during wait times.

Creating Your First Coroutine
import asyncio

# Define a coroutine with 'async def'
async def fetch_data():
    print("Fetching data...")
    await asyncio.sleep(2)  # Simulate I/O operation
    print("Data fetched!")
    return {"data": "Hello, Async World!"}

# Run the coroutine
async def main():
    result = await fetch_data()
    print(result)

# Execute
asyncio.run(main())
Expected Output:
Fetching data...
Data fetched!
{'data': 'Hello, Async World!'}
Key concepts:
  • async def - Defines a coroutine function
  • await - Pauses execution until the awaited operation completes
  • asyncio.run() - Runs the top-level coroutine (entry point)
  • asyncio.sleep() - Async version of time.sleep()

async/await: The Modern Syntax

Basic async/await pattern - the foundation of async Python.

import asyncio

async def greet(name):
    print(f"Hello, {name}!")
    await asyncio.sleep(1)  # Simulate async operation
    print(f"Goodbye, {name}!")
    return f"Greeted {name}"

async def main():
    # await suspends main() until greet() completes
    result = await greet("Alice")
    print(result)

asyncio.run(main())
Expected Output:
Hello, Alice!
Goodbye, Alice!
Greeted Alice

Running Tasks Concurrently

Python provides several ways to run multiple coroutines concurrently.

asyncio.gather() - Run Multiple Coroutines
import asyncio

async def fetch_page(url):
    print(f"Fetching {url}")
    await asyncio.sleep(2)  # Simulate network request
    return f"Content from {url}"

async def main():
    # gather runs all coroutines concurrently
    results = await asyncio.gather(
        fetch_page("https://example.com/page1"),
        fetch_page("https://example.com/page2"),
        fetch_page("https://example.com/page3")
    )

    for result in results:
        print(result)

asyncio.run(main())

# All three pages fetched in parallel
# Total time: ~2 seconds instead of 6!
Expected Output:
Fetching https://example.com/page1
Fetching https://example.com/page2
Fetching https://example.com/page3
Content from https://example.com/page1
Content from https://example.com/page2
Content from https://example.com/page3
asyncio.create_task() - Start Task Immediately
import asyncio

async def background_task(name):
    print(f"{name} started in background")
    await asyncio.sleep(3)
    print(f"{name} completed")
    return f"{name} result"

async def main():
    # create_task SCHEDULES the coroutine, it does not run it yet. main()
    # keeps going until it reaches an await, which is the only point where
    # the event loop can switch to the new tasks.
    task1 = asyncio.create_task(background_task("Task 1"))
    task2 = asyncio.create_task(background_task("Task 2"))

    # Do other work while tasks run
    print("Doing other work...")
    await asyncio.sleep(1)
    print("Still doing work...")

    # Wait for tasks to complete
    result1 = await task1
    result2 = await task2

    print(f"Results: {result1}, {result2}")

asyncio.run(main())

# Note the order below: "Doing other work..." prints before either task
# starts, because main() had not awaited anything yet.
Expected Output:
Doing other work...
Task 1 started in background
Task 2 started in background
Still doing work...
Task 1 completed
Task 2 completed
Results: Task 1 result, Task 2 result
asyncio.wait() - Advanced Control
import asyncio

async def task(name, delay):
    await asyncio.sleep(delay)
    return f"{name} done"

async def main():
    tasks = [
        asyncio.create_task(task("Fast", 1)),
        asyncio.create_task(task("Medium", 2)),
        asyncio.create_task(task("Slow", 3))
    ]

    # Wait for first task to complete
    done, pending = await asyncio.wait(
        tasks,
        return_when=asyncio.FIRST_COMPLETED
    )

    print(f"First task completed: {done.pop().result()}")
    print(f"Still running: {len(pending)} tasks")

    # Wait for all remaining
    done, pending = await asyncio.wait(pending)

    # asyncio.wait returns a set, which has no meaningful order. Sort the
    # results so the output is the same on every run.
    for result in sorted(t.result() for t in done):
        print(result)

asyncio.run(main())
Expected Output:
First task completed: Fast done
Still running: 2 tasks
Medium done
Slow done

Real-World Example: Concurrent API Requests

Here's a practical example of fetching data from multiple APIs concurrently.

import asyncio
import aiohttp  # Async HTTP library
import time

async def fetch_json(session, url):
    """Fetch JSON from a URL"""
    async with session.get(url) as response:
        return await response.json()

async def fetch_user_data(user_id):
    """Fetch user profile, posts, and comments concurrently"""
    base_url = "https://jsonplaceholder.typicode.com"

    async with aiohttp.ClientSession() as session:
        # Run all three requests concurrently
        user, posts, comments = await asyncio.gather(
            fetch_json(session, f"{base_url}/users/{user_id}"),
            fetch_json(session, f"{base_url}/posts?userId={user_id}"),
            fetch_json(session, f"{base_url}/comments?postId=1")
        )

        return {
            "user": user,
            "post_count": len(posts),
            "comment_count": len(comments)
        }

async def main():
    start = time.time()

    # Fetch data for multiple users concurrently
    results = await asyncio.gather(
        fetch_user_data(1),
        fetch_user_data(2),
        fetch_user_data(3)
    )

    # This one talks to a live API, so your own timing will differ
    elapsed = time.time() - start

    for i, result in enumerate(results, 1):
        print(f"User {i}: {result['post_count']} posts, "
              f"{result['comment_count']} comments")

    print(f"\nTotal time: {elapsed:.2f}s for 9 requests")

# Run
asyncio.run(main())
Expected Output:
User 1: 10 posts, 5 comments
User 2: 10 posts, 5 comments
User 3: 10 posts, 5 comments

Total time: 0.29s for 9 requests
Performance gain: all 9 requests (3 users × 3 endpoints) are in flight at once. Running the same 9 requests one after another against this API took 0.59s versus 0.26s concurrently, so about 2.5x rather than the 9x you might expect from the request count. The sequential version reuses one keep-alive connection from the same ClientSession, so it never pays for a fresh handshake, and each request after the first is cheap. The gap widens with slower endpoints, more of them, or a new connection per request.

Common Async Patterns

Common Pitfalls & Solutions

❌ Forgetting await
# Wrong - returns coroutine object
result = fetch_data()

# Right - awaits the result
result = await fetch_data()

Without await, you get a coroutine object, not the result.

❌ Blocking the Event Loop
import time

# Wrong - runs, but freezes the whole event loop for a second
time.sleep(1)

# Also wrong - time.sleep() returns None, and None cannot be awaited
await time.sleep(1)
# TypeError: object NoneType can't be used in 'await' expression

# Right - use the async version
await asyncio.sleep(1)

Never use blocking calls in async code. Use async alternatives.

❌ Not Running Concurrently
# Sequential (slow)
await task1()
await task2()

# Concurrent (fast)
await asyncio.gather(
    task1(), task2()
)

Use gather() or create_task() for concurrency.

❌ Mixing Sync/Async Code
# Can't call async from sync
def sync_func():
    result = await async_func()  # Error!

# Use asyncio.run() or run_until_complete()
def sync_func():
    result = asyncio.run(async_func())

Can't use await in regular functions.

Best Practices

Do This
  • Use asyncio.run() as entry point
  • Use async with for resources
  • Use asyncio.gather() for concurrency
  • Add timeouts to prevent hanging
  • Handle CancelledError properly
  • Use async libraries (aiohttp, asyncpg)
  • Keep coroutines focused and small
  • Use semaphores for rate limiting
Avoid This
  • Don't forget await keyword
  • Don't use blocking I/O (requests, time.sleep)
  • Don't create too many concurrent tasks
  • Don't ignore exceptions in tasks
  • Don't mix threading with asyncio
  • Don't use global event loops
  • Don't make coroutines too large
  • Don't forget error handling

Key Takeaways

  • Async is for I/O-bound operations - network, files, databases
  • Event loop manages execution - switches between tasks during waits
  • Coroutines are defined with async def - can be paused and resumed
  • await suspends execution - allows other tasks to run
  • asyncio.gather() runs tasks concurrently - massive performance gains
  • Use async libraries - aiohttp instead of requests, asyncpg instead of psycopg2
  • Don't block the event loop - never use time.sleep() or blocking I/O
  • Handle errors and cancellations - async code needs proper cleanup

Practice Exercises

Exercise 1: Concurrent Downloads

Create an async function that downloads multiple web pages concurrently using aiohttp. Measure the time difference between sequential and concurrent execution for 5 URLs. Add error handling and timeouts.

Exercise 2: Rate-Limited API Client

Build an async API client that respects rate limits using asyncio.Semaphore. Allow only 3 concurrent requests and implement retry logic with exponential backoff for failed requests.

Exercise 3: Async Web Scraper

Create an async web scraper that fetches a web page, extracts all links, then concurrently fetches all those linked pages. Limit concurrent requests to 5 and add a timeout of 10 seconds per request.

Additional Resources

  • Official Docs: docs.python.org/3/library/asyncio.html
  • aiohttp: Async HTTP client/server - docs.aiohttp.org
  • Real Python: Async IO in Python: A Complete Walkthrough
  • Book: "Using Asyncio in Python" by Caleb Hattingh
  • PEP 492: Coroutines with async and await syntax
  • asyncpg: Fast async PostgreSQL driver
What's Next?

You've learned async fundamentals! Now let's explore advanced async patterns and best practices for building production-ready asynchronous applications.

  • Async Context Managers - Resource management in async code
  • Async Iterators - Stream processing with async for loops
  • Error Handling - Robust error handling and timeout strategies