Webhooks and Event-Driven APIs

Build event-driven architectures with webhooks for real-time notifications and integrations

Push, Don't Pull

Webhooks reverse the API model. Instead of clients polling your API every few seconds asking "anything new?", your API proactively notifies clients when events occur. This is how Stripe tells you about successful payments, GitHub notifies you of new commits, and Slack alerts you of messages.

The problem with polling: If 1000 clients poll every 10 seconds, that's 100 requests/second wasted. 99% return "no changes". Webhooks reduce this to zero, you only send data when something actually happens.

In this lesson, you'll implement webhook endpoints, secure them with HMAC signatures, handle delivery failures with retry logic, manage webhook subscriptions, and follow production patterns from companies like Stripe, GitHub, and Twilio.

Lesson Sections

Webhooks vs Polling: Why Push Beats Pull

Polling means clients repeatedly ask "is there new data?" Most of the time, the answer is "no". Webhooks flip this, your API proactively pushes data to clients when events occur.

The Problem with Polling
# Client polling for new orders (inefficient)
import time
import requests

def poll_for_orders():
    """
    Check for new orders every 10 seconds.

    Problem: 99% of requests return "no new orders"
    Waste: 6 requests/minute * 1000 clients = 6000 req/min
    Cost: Server processing + database queries + bandwidth
    """
    last_order_id = 0

    while True:
        response = requests.get(
            f"https://api.example.com/orders?since={last_order_id}"
        )

        orders = response.json()

        if orders:
            print(f"New orders: {len(orders)}")
            for order in orders:
                process_order(order)
                last_order_id = order["id"]
        else:
            print("No new orders (wasted request)")

        # Wait 10 seconds and poll again
        time.sleep(10)


# Result with 1000 clients:
# - 6000 requests/minute to check for updates
# - 5940 return "no changes" (99% waste)
# - Latency: 0-10 seconds (average 5 seconds)
# - Database load: Constant queries even with no activity
Polling Problems
  • Wasted resources: 95-99% of polling requests return "no changes"
  • Latency: Average delay of half the polling interval (5s for 10s polling)
  • Scale issues: 1000 clients × 6 req/min = 6000 req/min doing nothing
  • Database load: Constant queries even when nothing happens
  • Cost: Server resources, bandwidth, database queries for no-ops
How Webhooks Work

With webhooks, clients register a callback URL. When an event occurs, your API makes an HTTP POST request to that URL with the event data. No polling required.

# Webhook flow (efficient)

# Step 1: Client registers webhook URL
POST /api/webhooks/subscribe
{
    "url": "https://client.example.com/webhook",
    "events": ["order.created", "order.shipped"]
}

# Response:
{
    "webhook_id": "wh_abc123",
    "status": "active"
}


# Step 2: Event occurs (order created)
# Server makes HTTP POST to client's webhook URL

POST https://client.example.com/webhook
Headers:
    Content-Type: application/json
    X-Webhook-Signature: abc123...
Body:
{
    "event": "order.created",
    "timestamp": "2024-01-15T10:30:00Z",
    "data": {
        "order_id": 12345,
        "customer": "john@example.com",
        "total": 99.99
    }
}

# Client receives webhook and responds immediately
HTTP/1.1 200 OK


# Result:
# - Zero polling requests (0 req/min instead of 6000!)
# - Instant notification (0 latency instead of 0-10s)
# - No database queries until event occurs
# - Scales effortlessly (1 request per event, not per client)
Webhook Benefits
  • Real-time: Instant notifications (no polling delay)
  • Efficient: 100× fewer requests (only when events occur)
  • Scalable: 1 request per event regardless of client count
  • Cost-effective: No wasted bandwidth, CPU, or database queries
  • Simple client: Just implement an HTTP endpoint, no polling loop
When to Use Webhooks vs Polling
ScenarioUse WebhooksUse Polling
Event frequencyInfrequent events (orders, payments, commits)Constant updates (stock prices, live scores)
Real-time needsInstant notification requiredSlight delay acceptable
Client countMany clients (reduces load 100×)Few clients (polling overhead minimal)
Client infrastructureClient can receive HTTP requestsClient behind firewall (can't receive requests)
ExamplesPayment notifications, CI/CD triggers, chat messagesStatus checks, simple dashboards, background sync

Creating Webhook Endpoints in FastAPI

Webhook endpoints are HTTP endpoints that receive event notifications. Clients provide their URL, and your API sends POST requests when events occur.

Basic Webhook Delivery System
from fastapi import FastAPI, BackgroundTasks
from pydantic import BaseModel, HttpUrl
import requests
from typing import List
import time

app = FastAPI()

# In-memory storage (use database in production)
webhooks = []


class WebhookSubscription(BaseModel):
    url: HttpUrl
    events: List[str]  # e.g., ["order.created", "order.shipped"]


class OrderEvent(BaseModel):
    event: str
    timestamp: str
    data: dict


@app.post("/api/webhooks/subscribe")
async def subscribe_webhook(subscription: WebhookSubscription):
    """
    Subscribe to webhook events.

    Client provides URL where they want to receive event notifications.
    """
    webhook = {
        "id": f"wh_{int(time.time())}",
        "url": str(subscription.url),
        "events": subscription.events,
        "status": "active",
        "created_at": time.time()
    }

    webhooks.append(webhook)

    return {
        "webhook_id": webhook["id"],
        "status": "active",
        "subscribed_events": subscription.events
    }


def send_webhook(webhook_url: str, event_data: dict):
    """
    Send webhook notification to client's URL.

    Makes HTTP POST request with event data.
    """
    try:
        response = requests.post(
            webhook_url,
            json=event_data,
            headers={"Content-Type": "application/json"},
            timeout=5  # 5 second timeout
        )

        if response.status_code == 200:
            print(f"✓ Webhook delivered to {webhook_url}")
            return True
        else:
            print(f"✗ Webhook failed: {response.status_code}")
            return False

    except requests.exceptions.RequestException as e:
        print(f"✗ Webhook error: {e}")
        return False


def notify_webhooks(event_type: str, event_data: dict, background_tasks: BackgroundTasks):
    """
    Notify all subscribed webhooks about an event.

    Filters webhooks by event type and sends notifications in background.
    """
    event_payload = {
        "event": event_type,
        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "data": event_data
    }

    # Find webhooks subscribed to this event
    for webhook in webhooks:
        if event_type in webhook["events"] and webhook["status"] == "active":
            # Send webhook in background (non-blocking)
            background_tasks.add_task(
                send_webhook,
                webhook["url"],
                event_payload
            )


@app.post("/api/orders")
async def create_order(
    customer_email: str,
    total: float,
    background_tasks: BackgroundTasks
):
    """
    Create order and trigger webhook notifications.

    When order is created, notify all webhooks subscribed to 'order.created' event.
    """
    # Create order in database
    order = {
        "id": 12345,
        "customer": customer_email,
        "total": total,
        "status": "pending"
    }

    # Trigger webhook notifications
    notify_webhooks(
        event_type="order.created",
        event_data=order,
        background_tasks=background_tasks
    )

    return {
        "message": "Order created",
        "order": order,
        "webhooks_notified": True
    }


# Example flow:
# 1. Client subscribes: POST /api/webhooks/subscribe
#    Body: {"url": "https://client.com/webhook", "events": ["order.created"]}
#    Response: {"webhook_id": "wh_1234", "status": "active"}
#
# 2. Order created: POST /api/orders
#    Body: {"customer_email": "john@example.com", "total": 99.99}
#    Server sends webhook to https://client.com/webhook
#
# 3. Client receives:
#    POST https://client.com/webhook
#    Body: {"event": "order.created", "data": {...}}
#    Client responds: 200 OK
Result: Real-Time Order Notifications

Server logs:

INFO: Order 12345 created
INFO: Notifying 3 webhooks subscribed to 'order.created'
INFO: ✓ Webhook delivered to https://client1.com/webhook
INFO: ✓ Webhook delivered to https://client2.com/webhook
INFO: ✓ Webhook delivered to https://client3.com/webhook
INFO: All webhooks delivered successfully
Implementing a Webhook Receiver (Client Side)

Clients need an HTTP endpoint to receive webhook notifications. Here's a simple FastAPI webhook receiver.

from fastapi import FastAPI, Request
from pydantic import BaseModel

app = FastAPI()


class WebhookPayload(BaseModel):
    event: str
    timestamp: str
    data: dict


@app.post("/webhook")
async def receive_webhook(payload: WebhookPayload):
    """
    Receive webhook from external API.

    Important:
    - Respond quickly (< 5 seconds) with 200 OK
    - Process event asynchronously if needed
    - Don't make slow API calls here
    """
    print(f"Received webhook: {payload.event}")

    # Route to appropriate handler based on event type
    if payload.event == "order.created":
        handle_order_created(payload.data)
    elif payload.event == "order.shipped":
        handle_order_shipped(payload.data)
    elif payload.event == "order.cancelled":
        handle_order_cancelled(payload.data)
    else:
        print(f"Unknown event: {payload.event}")

    # Always return 200 OK quickly
    return {"status": "received"}


def handle_order_created(order_data: dict):
    """Process new order event"""
    print(f"New order: {order_data['id']}")
    # Send email notification
    # Update inventory
    # Log to analytics


def handle_order_shipped(order_data: dict):
    """Process order shipped event"""
    print(f"Order shipped: {order_data['id']}")
    # Send tracking email
    # Update order status


def handle_order_cancelled(order_data: dict):
    """Process order cancelled event"""
    print(f"Order cancelled: {order_data['id']}")
    # Refund payment
    # Restore inventory


# Webhook receiver logs:
# INFO: Received webhook: order.created
# INFO: New order: 12345
# INFO: Sending confirmation email to john@example.com
# INFO: Response sent: 200 OK (35ms)

Securing Webhooks with HMAC Signatures

Problem: Anyone can send fake webhooks to your endpoint. Without verification, an attacker could send malicious events like "order.created" with fake data. Solution: HMAC signatures prove webhooks come from the authentic source.

How HMAC Signatures Work
HMAC Signature Flow
  1. Setup: Server generates secret key, shares with client (e.g., "sk_abc123...")
  2. Signing: Server computes HMAC of webhook payload using secret key
  3. Delivery: Server includes signature in X-Webhook-Signature header
  4. Verification: Client recomputes HMAC with same secret, compares signatures
  5. Result: Matching signatures = authentic webhook, mismatch = reject as fake
Implementing HMAC Signing (Server Side)
import hmac
import hashlib
import secrets
import json
import requests
from typing import Optional


def generate_webhook_secret() -> str:
    """
    Generate secure webhook secret key.

    Store this securely and share with client.
    """
    return secrets.token_urlsafe(32)


def sign_webhook_payload(payload: dict, secret: str) -> str:
    """
    Generate HMAC signature for webhook payload.

    Args:
        payload: Webhook event data
        secret: Shared secret key

    Returns:
        HMAC signature (hex string)
    """
    # Convert payload to JSON bytes
    payload_bytes = json.dumps(payload, sort_keys=True).encode('utf-8')

    # Compute HMAC-SHA256
    signature = hmac.new(
        key=secret.encode('utf-8'),
        msg=payload_bytes,
        digestmod=hashlib.sha256
    ).hexdigest()

    return signature


def send_signed_webhook(webhook_url: str, event_data: dict, secret: str) -> bool:
    """
    Send webhook with HMAC signature.

    Includes signature in X-Webhook-Signature header.
    """
    # Create payload
    payload = {
        "event": event_data["event"],
        "timestamp": event_data["timestamp"],
        "data": event_data["data"]
    }

    # Generate signature
    signature = sign_webhook_payload(payload, secret)

    try:
        response = requests.post(
            webhook_url,
            json=payload,
            headers={
                "Content-Type": "application/json",
                "X-Webhook-Signature": signature  # Include signature
            },
            timeout=5
        )

        return response.status_code == 200

    except requests.exceptions.RequestException as e:
        print(f"Webhook delivery failed: {e}")
        return False


# Example usage
webhook_secret = generate_webhook_secret()
print(f"Webhook secret (share with client): {webhook_secret}")

event_data = {
    "event": "order.created",
    "timestamp": "2024-01-15T10:30:00Z",
    "data": {"order_id": 12345, "total": 99.99}
}

# Send webhook with signature
success = send_signed_webhook(
    webhook_url="https://client.com/webhook",
    event_data=event_data,
    secret=webhook_secret
)

# Webhook includes header:
# X-Webhook-Signature: a7f2c8e1b9d4c5a8e3f9b1c2d3e4f5a6...
Verifying HMAC Signatures (Client Side)
from fastapi import FastAPI, Request, HTTPException, status
import hmac
import hashlib
import json

app = FastAPI()

# Store webhook secret securely (from server during subscription)
WEBHOOK_SECRET = "sk_abc123..."  # Get from server when subscribing


def verify_webhook_signature(payload: bytes, signature: str, secret: str) -> bool:
    """
    Verify webhook signature to ensure authenticity.

    Args:
        payload: Raw request body (bytes)
        signature: Signature from X-Webhook-Signature header
        secret: Shared secret key

    Returns:
        True if signature valid, False otherwise
    """
    # Compute expected signature
    expected_signature = hmac.new(
        key=secret.encode('utf-8'),
        msg=payload,
        digestmod=hashlib.sha256
    ).hexdigest()

    # Constant-time comparison (prevents timing attacks)
    return hmac.compare_digest(signature, expected_signature)


@app.post("/webhook")
async def receive_webhook(request: Request):
    """
    Receive and verify webhook.

    Security:
    - Checks X-Webhook-Signature header
    - Recomputes HMAC and compares
    - Rejects mismatched signatures
    """
    # Get signature from header
    signature = request.headers.get("X-Webhook-Signature")

    if not signature:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Missing X-Webhook-Signature header"
        )

    # Get raw body
    body = await request.body()

    # Verify signature
    if not verify_webhook_signature(body, signature, WEBHOOK_SECRET):
        print("⚠️ Webhook signature verification FAILED")
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid webhook signature"
        )

    print("✓ Webhook signature verified")

    # Parse payload
    payload = json.loads(body)

    # Process event
    print(f"Processing verified event: {payload['event']}")

    return {"status": "received"}


# Example logs:
#
# Authentic webhook:
# INFO: ✓ Webhook signature verified
# INFO: Processing verified event: order.created
# INFO: Response: 200 OK
#
# Fake webhook (no signature):
# WARNING: Missing X-Webhook-Signature header
# INFO: Response: 401 Unauthorized
#
# Fake webhook (wrong signature):
# WARNING: ⚠️ Webhook signature verification FAILED
# INFO: Response: 401 Unauthorized
Security Best Practices
  • Use HMAC-SHA256: Industry standard, secure hash algorithm
  • Constant-time comparison: Use hmac.compare_digest() to prevent timing attacks
  • Verify raw body: Compute HMAC on raw bytes before JSON parsing
  • Rotate secrets: Support multiple secrets for rotation without downtime
  • Store secrets securely: Use environment variables or secret managers, never commit to git
  • Always verify: Reject webhooks without valid signatures (prevent replay attacks)

Webhook Delivery and Retry Logic

Webhooks can fail. Client servers go down, networks timeout, endpoints return errors. Production webhook systems need retry logic with exponential backoff and dead-letter queues.

Implementing Retry Logic
from fastapi import FastAPI, BackgroundTasks
import requests
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
import time
import logging

app = FastAPI()
logger = logging.getLogger(__name__)


class WebhookDeliveryError(Exception):
    """Raised when webhook delivery fails"""
    pass


@retry(
    stop=stop_after_attempt(5),  # Try 5 times
    wait=wait_exponential(multiplier=60, min=60, max=3600),  # 1min, 2min, 4min, 8min, 16min
    retry=retry_if_exception_type(WebhookDeliveryError),
    before_sleep=lambda retry_state: logger.warning(
        f"Webhook delivery failed, retrying in {retry_state.next_action.sleep}s... "
        f"(attempt {retry_state.attempt_number}/5)"
    )
)
def deliver_webhook_with_retry(
    webhook_url: str,
    payload: dict,
    signature: str
) -> dict:
    """
    Deliver webhook with automatic retries.

    Retry schedule:
    - Attempt 1: Immediate
    - Attempt 2: Wait 1 minute
    - Attempt 3: Wait 2 minutes
    - Attempt 4: Wait 4 minutes
    - Attempt 5: Wait 8 minutes
    - If all fail: Raise exception

    Retries on:
    - Network errors
    - Timeouts
    - 5xx server errors

    Does NOT retry on:
    - 4xx client errors (indicates bad webhook URL or payload)
    """
    try:
        response = requests.post(
            webhook_url,
            json=payload,
            headers={
                "Content-Type": "application/json",
                "X-Webhook-Signature": signature
            },
            timeout=10
        )

        # Check response
        if response.status_code == 200:
            logger.info(f"✓ Webhook delivered successfully to {webhook_url}")
            return {
                "status": "delivered",
                "status_code": 200,
                "delivered_at": time.time()
            }

        elif 500 <= response.status_code < 600:
            # Server error - retry
            logger.warning(f"Server error {response.status_code}, will retry")
            raise WebhookDeliveryError(f"Server error: {response.status_code}")

        else:
            # Client error (4xx) - don't retry
            logger.error(f"Client error {response.status_code}, not retrying")
            return {
                "status": "failed",
                "status_code": response.status_code,
                "error": "Client error (permanent failure)"
            }

    except requests.exceptions.Timeout:
        logger.warning("Webhook timeout, will retry")
        raise WebhookDeliveryError("Timeout")

    except requests.exceptions.ConnectionError:
        logger.warning("Connection error, will retry")
        raise WebhookDeliveryError("Connection error")


async def send_webhook_async(
    webhook_url: str,
    event_data: dict,
    secret: str,
    background_tasks: BackgroundTasks
):
    """
    Send webhook asynchronously with retries.

    Runs in background task so it doesn't block API response.
    """
    payload = {
        "event": event_data["event"],
        "timestamp": event_data["timestamp"],
        "data": event_data["data"]
    }

    signature = sign_webhook_payload(payload, secret)

    # Add to background tasks
    background_tasks.add_task(
        deliver_webhook_with_retry,
        webhook_url,
        payload,
        signature
    )


@app.post("/api/orders")
async def create_order(
    customer_email: str,
    total: float,
    background_tasks: BackgroundTasks
):
    """Create order and send webhook with automatic retries"""

    order = {
        "id": 12345,
        "customer": customer_email,
        "total": total
    }

    event_data = {
        "event": "order.created",
        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "data": order
    }

    # Send webhook with retries (non-blocking)
    await send_webhook_async(
        webhook_url="https://client.com/webhook",
        event_data=event_data,
        secret="sk_abc123",
        background_tasks=background_tasks
    )

    return {"message": "Order created", "webhooks_queued": True}


# Example retry timeline:
# 10:00:00 - Attempt 1: Connection refused (immediate)
# 10:01:00 - Attempt 2: Timeout (wait 1 min)
# 10:03:00 - Attempt 3: 503 Server Error (wait 2 min)
# 10:07:00 - Attempt 4: 503 Server Error (wait 4 min)
# 10:15:00 - Attempt 5: 200 OK Success! (wait 8 min)
Retry Best Practices
  • Exponential backoff: Wait longer between retries (1min, 2min, 4min, 8min)
  • Max attempts: Give up after 5-10 attempts (prevents infinite retries)
  • Idempotency: Ensure webhooks can be delivered multiple times safely
  • Only retry transient failures: Retry network errors and 5xx, not 4xx client errors
  • Log everything: Track delivery attempts, failures, and final outcome
Dead Letter Queue for Failed Webhooks

When webhooks fail permanently (all retries exhausted), store them in a dead letter queuefor manual review and redelivery.

import redis
import json

redis_client = redis.Redis(host='localhost', port=6379, decode_responses=True)


def add_to_dead_letter_queue(webhook_url: str, payload: dict, error: str):
    """
    Store failed webhook in dead letter queue.

    Allows manual inspection and redelivery.
    """
    failed_webhook = {
        "webhook_url": webhook_url,
        "payload": payload,
        "error": error,
        "failed_at": time.time(),
        "attempts": 5
    }

    # Store in Redis list
    redis_client.lpush("webhook:dlq", json.dumps(failed_webhook))

    logger.error(f"Webhook permanently failed, added to DLQ: {webhook_url}")


@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=60, min=60, max=3600)
)
def deliver_webhook_with_dlq(
    webhook_url: str,
    payload: dict,
    signature: str
) -> dict:
    """
    Deliver webhook with retries and dead letter queue.

    If all retries fail, adds to DLQ instead of losing event.
    """
    try:
        # Attempt delivery with retries
        return deliver_webhook_with_retry(webhook_url, payload, signature)

    except Exception as e:
        # All retries exhausted
        logger.error(f"All retries failed: {e}")

        # Add to dead letter queue
        add_to_dead_letter_queue(webhook_url, payload, str(e))

        return {
            "status": "failed",
            "error": str(e),
            "added_to_dlq": True
        }


@app.post("/api/webhooks/dlq/retry/{index}")
async def retry_failed_webhook(index: int):
    """
    Manually retry webhook from dead letter queue.

    Admin can review failed webhooks and retry after fixing issues.
    """
    # Get failed webhook from DLQ
    failed_webhook_json = redis_client.lindex("webhook:dlq", index)

    if not failed_webhook_json:
        return {"error": "Webhook not found in DLQ"}

    failed_webhook = json.loads(failed_webhook_json)

    # Retry delivery
    result = deliver_webhook_with_retry(
        failed_webhook["webhook_url"],
        failed_webhook["payload"],
        sign_webhook_payload(failed_webhook["payload"], "sk_abc123")
    )

    if result["status"] == "delivered":
        # Remove from DLQ
        redis_client.lrem("webhook:dlq", 1, failed_webhook_json)
        return {"message": "Webhook redelivered successfully"}
    else:
        return {"error": "Retry failed", "result": result}


@app.get("/api/webhooks/dlq")
async def list_failed_webhooks():
    """List all webhooks in dead letter queue"""

    dlq_length = redis_client.llen("webhook:dlq")

    failed_webhooks = []
    for i in range(dlq_length):
        webhook_json = redis_client.lindex("webhook:dlq", i)
        failed_webhooks.append(json.loads(webhook_json))

    return {
        "total_failed": dlq_length,
        "webhooks": failed_webhooks
    }


# Example DLQ entry:
# {
#     "webhook_url": "https://client.com/webhook",
#     "payload": {"event": "order.created", "data": {...}},
#     "error": "Connection timeout after 5 attempts",
#     "failed_at": 1705320000,
#     "attempts": 5
# }

Webhook Subscription Management

Production webhook systems need full CRUD operations: create subscriptions, list active webhooks, update event filters, pause/resume delivery, and delete subscriptions.

Complete Subscription API
from fastapi import FastAPI, HTTPException, status
from pydantic import BaseModel, HttpUrl
from typing import List, Optional
import secrets
import time

app = FastAPI()

# Database (use SQLAlchemy/PostgreSQL in production)
webhooks_db = {}


class WebhookCreate(BaseModel):
    url: HttpUrl
    events: List[str]
    description: Optional[str] = None


class WebhookUpdate(BaseModel):
    url: Optional[HttpUrl] = None
    events: Optional[List[str]] = None
    description: Optional[str] = None
    status: Optional[str] = None  # active, paused


@app.post("/api/webhooks", status_code=status.HTTP_201_CREATED)
async def create_webhook_subscription(webhook: WebhookCreate):
    """
    Create new webhook subscription.

    Returns webhook ID and secret for signature verification.
    """
    webhook_id = f"wh_{secrets.token_urlsafe(16)}"
    secret = f"whsec_{secrets.token_urlsafe(32)}"

    webhook_data = {
        "id": webhook_id,
        "url": str(webhook.url),
        "events": webhook.events,
        "description": webhook.description,
        "secret": secret,
        "status": "active",
        "created_at": time.time(),
        "updated_at": time.time(),
        "delivery_stats": {
            "total_delivered": 0,
            "total_failed": 0,
            "last_delivery": None
        }
    }

    webhooks_db[webhook_id] = webhook_data

    return {
        "id": webhook_id,
        "url": webhook_data["url"],
        "events": webhook_data["events"],
        "secret": secret,  # Client stores this for verification
        "status": "active"
    }


@app.get("/api/webhooks")
async def list_webhooks(status: Optional[str] = None):
    """
    List all webhook subscriptions.

    Query params:
    - status: Filter by status (active, paused)
    """
    webhooks = list(webhooks_db.values())

    if status:
        webhooks = [w for w in webhooks if w["status"] == status]

    # Don't return secrets in list
    return {
        "total": len(webhooks),
        "webhooks": [
            {
                "id": w["id"],
                "url": w["url"],
                "events": w["events"],
                "status": w["status"],
                "created_at": w["created_at"],
                "delivery_stats": w["delivery_stats"]
            }
            for w in webhooks
        ]
    }


@app.get("/api/webhooks/{webhook_id}")
async def get_webhook(webhook_id: str):
    """Get webhook subscription details"""

    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    return webhook


@app.patch("/api/webhooks/{webhook_id}")
async def update_webhook(webhook_id: str, update: WebhookUpdate):
    """
    Update webhook subscription.

    Can update:
    - URL
    - Event filters
    - Description
    - Status (active/paused)
    """
    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    # Update fields
    if update.url:
        webhook["url"] = str(update.url)
    if update.events:
        webhook["events"] = update.events
    if update.description:
        webhook["description"] = update.description
    if update.status:
        if update.status not in ["active", "paused"]:
            raise HTTPException(
                status_code=status.HTTP_400_BAD_REQUEST,
                detail="Status must be 'active' or 'paused'"
            )
        webhook["status"] = update.status

    webhook["updated_at"] = time.time()

    return {
        "id": webhook_id,
        "url": webhook["url"],
        "events": webhook["events"],
        "status": webhook["status"],
        "updated_at": webhook["updated_at"]
    }


@app.delete("/api/webhooks/{webhook_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_webhook(webhook_id: str):
    """Delete webhook subscription"""

    if webhook_id not in webhooks_db:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    del webhooks_db[webhook_id]

    return None  # 204 No Content


@app.post("/api/webhooks/{webhook_id}/pause")
async def pause_webhook(webhook_id: str):
    """Pause webhook delivery temporarily"""

    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    webhook["status"] = "paused"
    webhook["updated_at"] = time.time()

    return {"message": "Webhook paused", "status": "paused"}


@app.post("/api/webhooks/{webhook_id}/resume")
async def resume_webhook(webhook_id: str):
    """Resume paused webhook delivery"""

    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    webhook["status"] = "active"
    webhook["updated_at"] = time.time()

    return {"message": "Webhook resumed", "status": "active"}


@app.get("/api/webhooks/{webhook_id}/deliveries")
async def get_webhook_deliveries(webhook_id: str, limit: int = 100):
    """
    Get webhook delivery history.

    Shows recent deliveries with status, response code, timestamps.
    """
    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Webhook not found"
        )

    # In production, fetch from deliveries table
    deliveries = [
        {
            "id": "del_123",
            "status": "delivered",
            "status_code": 200,
            "delivered_at": time.time() - 3600,
            "duration_ms": 45
        },
        {
            "id": "del_124",
            "status": "failed",
            "status_code": 503,
            "delivered_at": time.time() - 1800,
            "error": "Server error",
            "retries": 3
        }
    ]

    return {
        "webhook_id": webhook_id,
        "total_deliveries": len(deliveries),
        "deliveries": deliveries[:limit]
    }
API Usage Example
# Create subscription
POST /api/webhooks
{"url": "https://client.com/webhook", "events": ["order.created"]}
→ {"id": "wh_abc123", "secret": "whsec_xyz789", "status": "active"}

# List subscriptions
GET /api/webhooks
→ {"total": 3, "webhooks": [...]}

# Pause temporarily
POST /api/webhooks/wh_abc123/pause
→ {"status": "paused"}

# Update event filters
PATCH /api/webhooks/wh_abc123
{"events": ["order.created", "order.shipped"]}
→ {"events": ["order.created", "order.shipped"]}

# Check delivery history
GET /api/webhooks/wh_abc123/deliveries
→ {"deliveries": [{"status": "delivered", "status_code": 200}]}

# Delete subscription
DELETE /api/webhooks/wh_abc123
→ 204 No Content

Event Payload Design

Well-designed webhook payloads are consistent, self-contained, and versioned. Study how Stripe, GitHub, and Twilio structure their webhook events for inspiration.

Webhook Payload Best Practices
# Good webhook payload structure (following Stripe pattern)
{
    "id": "evt_1A2B3C4D5E6F",  # Unique event ID
    "object": "event",  # Object type
    "api_version": "2024-01-15",  # API version for compatibility
    "created": 1705320000,  # Unix timestamp
    "type": "order.created",  # Event type (dot-separated)
    "data": {
        "object": {
            "id": "ord_123456",
            "object": "order",
            "customer": {
                "id": "cus_abc123",
                "email": "john@example.com",
                "name": "John Doe"
            },
            "items": [
                {
                    "id": "item_1",
                    "product": "Widget",
                    "quantity": 2,
                    "price": 1999  # Cents (avoid float precision issues)
                }
            ],
            "total": 3998,
            "currency": "usd",
            "status": "pending",
            "created": 1705320000,
            "metadata": {
                "order_source": "web",
                "campaign_id": "summer_sale"
            }
        },
        "previous_attributes": {}  # For update events
    },
    "livemode": true,  # Production vs test mode
    "pending_webhooks": 2,  # How many webhooks are queued
    "request": {
        "id": "req_xyz789",  # Original API request that triggered event
        "idempotency_key": "idp_abc123"
    }
}
Payload Design Principles
  • Include everything: Payload should be self-contained (no need for additional API calls)
  • Consistent structure: All events follow same format (id, type, data, created)
  • Unique event ID: For deduplication and idempotency
  • Timestamps: ISO 8601 or Unix timestamps for all time fields
  • Event types: Dot-separated hierarchy (order.created, order.updated, order.cancelled)
  • API version: Include version for backward compatibility
  • Test mode flag: Distinguish production vs test events
  • Previous attributes: For update events, show what changed
Event Types Hierarchy
Event TypeWhen TriggeredExample Use Case
order.createdNew order placedSend confirmation email
order.updatedOrder details changedUpdate inventory system
order.cancelledOrder cancelled by customer/adminProcess refund
order.fulfilledOrder shipped/completedSend tracking number
payment.succeededPayment processed successfullyFulfill order
payment.failedPayment declinedNotify customer to update card
customer.createdNew customer registeredAdd to CRM
customer.updatedCustomer profile changedSync with marketing tools
Implementing Event Types
from enum import Enum
from typing import Dict, Any
import time


class EventType(str, Enum):
    """Webhook event types"""
    ORDER_CREATED = "order.created"
    ORDER_UPDATED = "order.updated"
    ORDER_CANCELLED = "order.cancelled"
    ORDER_FULFILLED = "order.fulfilled"
    PAYMENT_SUCCEEDED = "payment.succeeded"
    PAYMENT_FAILED = "payment.failed"
    CUSTOMER_CREATED = "customer.created"
    CUSTOMER_UPDATED = "customer.updated"


def create_webhook_event(
    event_type: EventType,
    data: Dict[str, Any],
    previous_attributes: Dict[str, Any] = None
) -> Dict[str, Any]:
    """
    Create standardized webhook event payload.

    Args:
        event_type: Type of event
        data: Event data (the main object)
        previous_attributes: For update events, what changed

    Returns:
        Formatted webhook payload
    """
    event_id = f"evt_{secrets.token_urlsafe(16)}"

    payload = {
        "id": event_id,
        "object": "event",
        "api_version": "2024-01-15",
        "created": int(time.time()),
        "type": event_type.value,
        "data": {
            "object": data,
            "previous_attributes": previous_attributes or {}
        },
        "livemode": True  # Set based on environment
    }

    return payload


# Example: Order created event
order_data = {
    "id": "ord_123456",
    "object": "order",
    "customer": {
        "id": "cus_abc123",
        "email": "john@example.com",
        "name": "John Doe"
    },
    "total": 9999,
    "currency": "usd",
    "status": "pending",
    "created": int(time.time())
}

event = create_webhook_event(EventType.ORDER_CREATED, order_data)


# Example: Order updated event (with previous attributes)
updated_order = {
    "id": "ord_123456",
    "status": "shipped",  # Changed from "pending"
    "tracking_number": "1Z999AA10123456784"  # New field
}

previous = {
    "status": "pending"  # What changed
}

event = create_webhook_event(
    EventType.ORDER_UPDATED,
    updated_order,
    previous
)

# Payload shows:
# - Current state in data.object
# - What changed in data.previous_attributes
# Client can see: "status changed from pending to shipped"

Testing Webhooks Locally

Testing webhooks is tricky because your local development server isn't publicly accessible. Solutions: ngrok for tunneling, webhook testing tools, and local mock servers.

Option 1: ngrok (Expose Local Server)
# Step 1: Install ngrok
# Download from https://ngrok.com or:
brew install ngrok  # macOS
# or
snap install ngrok  # Linux

# Step 2: Start your FastAPI server locally
uvicorn main:app --host 0.0.0.0 --port 8000

# Step 3: Create ngrok tunnel
ngrok http 8000

# Output:
# Forwarding: https://abc123.ngrok.io -> http://localhost:8000

# Step 4: Use ngrok URL for webhook subscription
POST https://api.example.com/api/webhooks/subscribe
{
    "url": "https://abc123.ngrok.io/webhook",  # Public ngrok URL
    "events": ["order.created"]
}

# Step 5: Trigger event
POST https://api.example.com/api/orders
{"customer": "john@example.com", "total": 99.99}

# Step 6: Webhook delivered to your local server!
# Your local FastAPI receives:
POST http://localhost:8000/webhook
{"event": "order.created", "data": {...}}

# ngrok dashboard (http://localhost:4040) shows all webhook requests
ngrok Benefits
  • Instant public URL: localhost:8000 becomes https://abc123.ngrok.io
  • Inspect requests: Dashboard shows all webhook payloads and headers
  • Replay webhooks: Re-send previous webhooks for testing
  • No deployment: Test webhooks without deploying to staging server
Option 2: Manual Webhook Trigger (Test Endpoint)

Create a test endpoint that manually triggers webhooks for development and debugging.

from fastapi import FastAPI

app = FastAPI()


@app.post("/api/webhooks/test")
async def trigger_test_webhook(
    webhook_id: str,
    event_type: str,
    test_data: dict = None
):
    """
    Manually trigger webhook for testing.

    Useful for:
    - Local development
    - Integration testing
    - Debugging webhook handlers
    """
    webhook = webhooks_db.get(webhook_id)

    if not webhook:
        raise HTTPException(404, "Webhook not found")

    # Create test event
    if not test_data:
        # Use sample data
        test_data = {
            "id": "test_123",
            "sample": True,
            "message": "This is a test webhook"
        }

    event = create_webhook_event(
        event_type=event_type,
        data=test_data
    )

    # Deliver immediately (synchronous for testing)
    signature = sign_webhook_payload(event, webhook["secret"])

    result = deliver_webhook_with_retry(
        webhook["url"],
        event,
        signature
    )

    return {
        "message": "Test webhook sent",
        "event": event,
        "result": result
    }


# Usage:
POST /api/webhooks/test
{
    "webhook_id": "wh_abc123",
    "event_type": "order.created",
    "test_data": {
        "id": "test_order_1",
        "total": 9999
    }
}

# Response:
{
    "message": "Test webhook sent",
    "event": {...},
    "result": {"status": "delivered", "status_code": 200}
}
Option 3: Mock Webhook Server

Create a mock webhook receiver to test webhook sending without a real client.

# mock_webhook_server.py
from fastapi import FastAPI, Request
import json
import time

app = FastAPI()

received_webhooks = []


@app.post("/webhook")
async def mock_webhook_receiver(request: Request):
    """
    Mock webhook receiver for testing.

    Logs all received webhooks for inspection.
    """
    # Get headers
    headers = dict(request.headers)

    # Get body
    body = await request.json()

    # Store webhook
    webhook_data = {
        "timestamp": time.time(),
        "headers": headers,
        "body": body
    }

    received_webhooks.append(webhook_data)

    print("=" * 60)
    print("WEBHOOK RECEIVED")
    print("=" * 60)
    print(f"Event: {body.get('event')}")
    print(f"Signature: {headers.get('x-webhook-signature', 'MISSING')}")
    print(f"Body: {json.dumps(body, indent=2)}")
    print("=" * 60)

    return {"status": "received"}


@app.get("/webhooks/received")
async def list_received_webhooks():
    """View all received webhooks"""
    return {
        "total": len(received_webhooks),
        "webhooks": received_webhooks
    }


@app.delete("/webhooks/received")
async def clear_received_webhooks():
    """Clear webhook history"""
    received_webhooks.clear()
    return {"message": "Cleared"}


# Run mock server:
# uvicorn mock_webhook_server:app --port 8001

# Subscribe to mock server:
POST /api/webhooks/subscribe
{"url": "http://localhost:8001/webhook", "events": ["order.created"]}

# View received webhooks:
GET http://localhost:8001/webhooks/received

Production Best Practices

Production webhook systems require careful consideration of security, reliability, monitoring, and documentation. Learn from industry leaders like Stripe, GitHub, and Twilio.

Security Checklist
Security Best Practices
  • ✓ HMAC signatures: Always sign webhooks with HMAC-SHA256
  • ✓ HTTPS only: Reject webhook URLs that aren't HTTPS
  • ✓ Verify signatures: Clients must verify signatures before processing
  • ✓ Rotate secrets: Support multiple active secrets for rotation
  • ✓ Rate limit subscriptions: Limit webhooks per account (prevent abuse)
  • ✓ Validate URLs: Block localhost, internal IPs, localhost, 169.254.x.x
  • ✓ Timeout quickly: 5-10 second timeout (don't wait forever)
  • ✓ User-Agent header: Identify webhooks (e.g., "YourAPI-Webhooks/1.0")
Reliability Checklist
Reliability Best Practices
  • ✓ Retry logic: Exponential backoff with 5-10 attempts
  • ✓ Dead letter queue: Store failed webhooks for manual review
  • ✓ Idempotent delivery: Include event ID for deduplication
  • ✓ Async delivery: Use background tasks or queue (don't block API)
  • ✓ Circuit breaker: Pause webhooks after repeated failures
  • ✓ Delivery history: Track all attempts, responses, timing
  • ✓ Manual retry: Allow admins to retry failed webhooks
  • ✓ Webhook testing: Provide test mode and trigger endpoints
Monitoring and Observability
from prometheus_client import Counter, Histogram, Gauge

# Webhook metrics
WEBHOOK_DELIVERIES = Counter(
    'webhook_deliveries_total',
    'Total webhook deliveries',
    ['event_type', 'status']  # labels: order.created, delivered/failed
)

WEBHOOK_DELIVERY_DURATION = Histogram(
    'webhook_delivery_duration_seconds',
    'Webhook delivery duration',
    ['event_type'],
    buckets=[0.1, 0.5, 1.0, 2.0, 5.0, 10.0]
)

WEBHOOK_QUEUE_SIZE = Gauge(
    'webhook_queue_size',
    'Webhooks waiting for delivery'
)

WEBHOOK_FAILURES = Counter(
    'webhook_failures_total',
    'Failed webhook deliveries',
    ['event_type', 'error_type']  # timeout, server_error, client_error
)


def deliver_webhook_with_metrics(url: str, payload: dict, signature: str):
    """Deliver webhook with metric tracking"""

    event_type = payload.get('type', 'unknown')

    with WEBHOOK_DELIVERY_DURATION.labels(event_type=event_type).time():
        try:
            response = requests.post(url, json=payload, headers={...}, timeout=5)

            if response.status_code == 200:
                WEBHOOK_DELIVERIES.labels(
                    event_type=event_type,
                    status='delivered'
                ).inc()
                return True
            else:
                WEBHOOK_DELIVERIES.labels(
                    event_type=event_type,
                    status='failed'
                ).inc()

                if 400 <= response.status_code < 500:
                    error_type = 'client_error'
                else:
                    error_type = 'server_error'

                WEBHOOK_FAILURES.labels(
                    event_type=event_type,
                    error_type=error_type
                ).inc()

                return False

        except requests.exceptions.Timeout:
            WEBHOOK_FAILURES.labels(
                event_type=event_type,
                error_type='timeout'
            ).inc()
            return False


# Grafana dashboard queries:
# - Delivery success rate:
#   sum(rate(webhook_deliveries_total{status="delivered"}[5m])) /
#   sum(rate(webhook_deliveries_total[5m]))
#
# - P95 delivery latency:
#   histogram_quantile(0.95, rate(webhook_delivery_duration_seconds_bucket[5m]))
#
# - Failure rate by error type:
#   sum by (error_type) (rate(webhook_failures_total[5m]))
Documentation Requirements
Document for Developers

Essential webhook documentation:

  • Getting started guide: How to create webhook subscription
  • Event types reference: Complete list of all event types with examples
  • Payload structure: Schema for all webhook payloads
  • Signature verification: How to verify HMAC signatures (code examples)
  • Retry behavior: When/how webhooks are retried
  • Testing guide: How to test webhooks locally (ngrok, test endpoints)
  • Troubleshooting: Common issues and solutions
  • Best practices: Idempotency, quick responses, async processing
Real-World Examples
CompanyWebhook Use CasesDocumentation
StripePayment succeeded/failed, subscription events, dispute notificationsstripe.com/docs/webhooks
GitHubPush events, pull requests, issues, releases, CI/CD triggersdocs.github.com/webhooks
TwilioSMS received, call status, delivery receiptstwilio.com/docs/proxy/api/webhooks
ShopifyOrder created, product updated, customer registeredshopify.dev/docs/api/webhooks/latest
SlackMessages posted, reactions added, files uploadedapi.slack.com/messaging/webhooks