WebSockets and Real-Time APIs
Build real-time, bidirectional communication for chat, notifications, and live updates
Real-Time, Bidirectional Communication
HTTP is request-response. Client asks, server answers. But what if the server needs to push data to the client immediately? Chat messages, stock prices, game state, collaborative editing - these need real-time bidirectional communication.
WebSockets create a persistent connection where both client and server can send messages anytime. No polling, no overhead, just instant two-way communication. This is how Slack sends messages, Google Docs syncs changes, and trading platforms update prices.
In this lesson, you'll master the WebSocket protocol, implement chat servers with FastAPI, authenticate connections, manage rooms and broadcasting, explore Server-Sent Events (SSE) as a simpler alternative, and learn production patterns for scaling WebSocket servers.
Lesson Sections
WebSockets vs HTTP: When to Use Each
HTTP is stateless and unidirectional. Each request opens a new connection, gets a response, and closes. WebSockets maintain a persistent, bidirectional connection. Understanding when to use each is crucial.
HTTP Request-Response Model
# HTTP: Client initiates, server responds, connection closes
# Request 1
Client → Server: GET /api/messages
Server → Client: 200 OK [message1, message2]
Connection: CLOSED
# Request 2 (new connection)
Client → Server: GET /api/messages
Server → Client: 200 OK [message1, message2, message3]
Connection: CLOSED
# Request 3 (new connection)
Client → Server: POST /api/messages {text: "Hello"}
Server → Client: 201 Created {id: 123}
Connection: CLOSED
# Limitations:
# - Server can't push data to client
# - Client must poll for updates
# - New TCP connection for each request (overhead)
# - Headers sent every time (Cookie, User-Agent, etc.)
# - Latency: connection setup + request + responseWebSocket Bidirectional Model
# WebSocket: Persistent connection, both sides can send anytime
# Handshake (HTTP upgrade)
Client → Server: GET /ws (Upgrade: websocket)
Server → Client: 101 Switching Protocols
Connection: ESTABLISHED (stays open)
# Bidirectional communication (no connection overhead)
Client → Server: {"action": "join", "room": "chat1"}
Server → Client: {"type": "joined", "room": "chat1"}
Server → Client: {"type": "message", "text": "New message!", "from": "Alice"}
Server → Client: {"type": "message", "text": "Another message!", "from": "Bob"}
Client → Server: {"action": "send", "text": "Hello everyone!"}
Server → Client: {"type": "message", "text": "Hello everyone!", "from": "You"}
# Connection stays open until explicitly closed
# No reconnection overhead
# Instant message delivery (no polling)
# Minimal bandwidth (no repeated headers)Performance Comparison
Chat application with 1000 users, 10 messages/minute:
| Metric | HTTP Polling (10s interval) | WebSockets |
|---|---|---|
| Requests/min | 6000 (1000 users × 6 polls) | 10 (actual messages only) |
| Bandwidth | ~60 MB/min (headers + body) | ~10 KB/min (minimal frames) |
| Latency | 0-10s (average 5s delay) | <100ms (instant) |
| Result | 600× more requests | 6000× less bandwidth |
When to Use Each Protocol
Use WebSockets For
- Chat applications
- Live notifications
- Collaborative editing (Google Docs)
- Real-time dashboards
- Online gaming
- Live sports scores
- Stock tickers
- IoT device updates
- Video/audio streaming metadata
Use HTTP For
- REST APIs (CRUD operations)
- Data fetching/submission
- File uploads/downloads
- Authentication
- Search queries
- Form submissions
- Stateless operations
- Caching needed
- Mobile apps (battery conscious)
WebSocket Protocol and Handshake
WebSockets start as HTTP then "upgrade" to the WebSocket protocol. Understanding the handshake process helps debug connection issues and implement custom protocols.
The WebSocket Handshake
# Step 1: Client initiates WebSocket handshake (HTTP request) GET /ws HTTP/1.1 Host: api.example.com Upgrade: websocket # Request protocol upgrade Connection: Upgrade # Connection should be upgraded Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== # Random base64 key Sec-WebSocket-Version: 13 # WebSocket protocol version Origin: https://example.com # CORS origin # Step 2: Server accepts upgrade (HTTP 101 response) HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo= # Hashed key for validation # Step 3: Connection upgraded to WebSocket # - HTTP is done, WebSocket protocol takes over # - Both sides can now send frames anytime # - Connection stays open until explicitly closed # WebSocket frame structure (simplified): # ┌─────────┬──────────┬─────────┬──────────┐ # │ FIN=1 │ Opcode │ Payload │ Data │ # │ (final) │ (text/ │ Length │ (actual │ # │ │ binary) │ │ message) │ # └─────────┴──────────┴─────────┴──────────┘ # Opcode types: # 0x1 = Text frame # 0x2 = Binary frame # 0x8 = Close frame # 0x9 = Ping frame # 0xA = Pong frame (heartbeat)
How Sec-WebSocket-Accept Works
The server proves it understands WebSockets by computing a response from the client's key:
- Take
Sec-WebSocket-Keyfrom client - Append magic string:
258EAFA5-E914-47DA-95CA-C5AB0DC85B11 - Compute SHA-1 hash
- Base64 encode result
- Send as
Sec-WebSocket-Acceptheader
WebSocket Message Types
| Message Type | Purpose | Example |
|---|---|---|
| Text (0x1) | UTF-8 text messages (JSON, plain text) | {"type": "chat"} |
| Binary (0x2) | Binary data (images, files, compressed data) | Image bytes, protobuf |
| Close (0x8) | Close connection gracefully with reason code | 1000 = Normal closure |
| Ping (0x9) | Keepalive / check if connection is alive | Server sends periodically |
| Pong (0xA) | Response to ping (automatic) | Client responds to ping |
WebSocket Close Codes
| Code | Name | Meaning |
|---|---|---|
1000 | Normal Closure | Connection closed normally (user logged out) |
1001 | Going Away | Server shutting down or client navigating away |
1002 | Protocol Error | Invalid WebSocket frame received |
1003 | Unsupported Data | Received data type not supported |
1006 | Abnormal Closure | Connection lost (no close frame sent) |
1008 | Policy Violation | Message violates policy (e.g., too large) |
1011 | Server Error | Unexpected condition prevented fulfillment |
4000-4999 | Custom Codes | Application-specific close reasons |
Implementing WebSockets in FastAPI
FastAPI has built-in WebSocket support with async/await syntax. Creating a WebSocket endpoint is as simple as defining a function that accepts a WebSocket parameter.
Basic WebSocket Echo Server
Server Implementation:
# main.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
app = FastAPI()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
"""
Simple echo server - sends back whatever it receives.
Flow:
1. Accept connection
2. Receive message
3. Echo it back
4. Repeat until client disconnects
"""
# Accept the WebSocket connection
await websocket.accept()
print("Client connected")
try:
while True:
# Wait for message from client
data = await websocket.receive_text()
print(f"Received: {data}")
# Echo message back to client
await websocket.send_text(f"Echo: {data}")
except WebSocketDisconnect:
print("Client disconnected")Run the server:
uvicorn main:app --reload
Client Implementation:
First, install the websockets library:
pip install websockets
Then create the client:
# client.py
import asyncio
import websockets
async def test_echo():
uri = "ws://localhost:8000/ws"
async with websockets.connect(uri) as websocket:
# Send message
await websocket.send("Hello WebSocket!")
# Receive echo
response = await websocket.recv()
print(f"Received: {response}") # "Echo: Hello WebSocket!"
asyncio.run(test_echo())Result: Echo Server
# Server logs: INFO: Client connected INFO: Received: Hello WebSocket! INFO: Received: How are you? INFO: Received: Goodbye! INFO: Client disconnected # Client receives: Echo: Hello WebSocket! Echo: How are you? Echo: Goodbye!
Chat Server with Multiple Clients
Real applications need to manage multiple concurrent WebSocket connections and broadcast messages to all connected clients.
Server Implementation:
# chat_server.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from typing import List
import json
from datetime import datetime, timezone
app = FastAPI()
class ConnectionManager:
"""Manage WebSocket connections and broadcasting"""
def __init__(self):
self.active_connections: List[WebSocket] = []
async def connect(self, websocket: WebSocket):
"""Accept and store new connection"""
await websocket.accept()
self.active_connections.append(websocket)
print(f"Client connected. Total connections: {len(self.active_connections)}")
def disconnect(self, websocket: WebSocket):
"""Remove connection from active list"""
self.active_connections.remove(websocket)
print(f"Client disconnected. Total connections: {len(self.active_connections)}")
async def broadcast(self, message: dict):
"""Send message to all connected clients"""
for connection in self.active_connections:
try:
await connection.send_json(message)
except Exception as e:
print(f"Error sending to client: {e}")
manager = ConnectionManager()
@app.websocket("/ws/chat")
async def chat_endpoint(websocket: WebSocket):
"""
Chat server supporting multiple clients.
Features:
- Broadcasts messages to all connected clients
- Shows join/leave notifications
- Includes timestamp and message ID
"""
await manager.connect(websocket)
# Notify all clients about new user
await manager.broadcast({
"type": "user_joined",
"message": "A user joined the chat",
"timestamp": datetime.now(timezone.utc).isoformat(),
"total_users": len(manager.active_connections)
})
try:
while True:
# Receive message from client
data = await websocket.receive_text()
# Parse message
try:
message_data = json.loads(data)
except json.JSONDecodeError:
# Plain text message
message_data = {"text": data}
# Broadcast to all clients
broadcast_message = {
"type": "message",
"text": message_data.get("text", ""),
"username": message_data.get("username", "Anonymous"),
"timestamp": datetime.now(timezone.utc).isoformat()
}
await manager.broadcast(broadcast_message)
except WebSocketDisconnect:
manager.disconnect(websocket)
# Notify all clients about user leaving
await manager.broadcast({
"type": "user_left",
"message": "A user left the chat",
"timestamp": datetime.now(timezone.utc).isoformat(),
"total_users": len(manager.active_connections)
})Run the server:
uvicorn chat_server:app --reload
Client Implementation:
Install the websockets library if you haven't already:
pip install websockets
Then create the chat client:
# chat_client.py
import asyncio
import json
import websockets
async def chat_client(username: str):
uri = "ws://localhost:8000/ws/chat"
try:
async with websockets.connect(uri) as websocket:
# 1. Background task to receive messages
async def receive_messages():
try:
while True:
message = await websocket.recv()
data = json.loads(message)
# We use \r to clear the "You: " line momentarily
print(f"\r[{data['type']}] {data.get('text', data.get('message'))}\nYou: ", end="")
except websockets.ConnectionClosed:
print("\nConnection closed by server.")
asyncio.create_task(receive_messages())
# 2. Non-blocking input loop
while True:
# Use run_in_executor to run the blocking input() in a thread
text = await asyncio.get_event_loop().run_in_executor(
None, lambda: input("You: ")
)
if text.lower() == 'exit':
break
message = json.dumps({"username": username, "text": text})
await websocket.send(message)
except Exception as e:
print(f"Failed to connect: {e}")Run multiple clients in separate terminals:
# Terminal 1
python -c "import asyncio; from chat_client import chat_client; asyncio.run(chat_client('Alice'))"
# Terminal 2
python -c "import asyncio; from chat_client import chat_client; asyncio.run(chat_client('Bob'))"
# Terminal 3
python -c "import asyncio; from chat_client import chat_client; asyncio.run(chat_client('Charlie'))"Result: Multi-User Chat
Server logs:
Client connected. Total connections: 1
Broadcasting: {"type": "user_joined", "total_users": 1}
Client connected. Total connections: 2
Broadcasting: {"type": "user_joined", "total_users": 2}
Broadcasting: {"type": "message", "text": "Hello everyone!", "username": "Alice"}
Broadcasting: {"type": "message", "text": "Hi Alice!", "username": "Bob"}
Client disconnected. Total connections: 1
Broadcasting: {"type": "user_left", "total_users": 1}Client 1 (Alice) sees:
[user_joined] A user joined the chat [user_joined] A user joined the chat You: Hello everyone! [message] Hello everyone! [message] Hi Alice! [user_left] A user left the chat
Handling Binary Data
WebSockets can send binary data like images, audio, or compressed payloads. Useful for file transfers or efficient protocols (e.g., Protocol Buffers).
Server Implementation:
First, install the required dependencies:
pip install fastapi pillow uvicorn
Then create the server:
# image_server.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
import io
from PIL import Image
app = FastAPI()
@app.websocket("/ws/images")
async def image_websocket(websocket: WebSocket):
"""
WebSocket endpoint for binary image data.
Receives images, processes them, sends back.
"""
await websocket.accept()
try:
while True:
# Receive binary data
image_bytes = await websocket.receive_bytes()
print(f"Received {len(image_bytes)} bytes")
# Process image (example: resize)
image = Image.open(io.BytesIO(image_bytes))
resized = image.resize((100, 100))
# Convert back to bytes
output = io.BytesIO()
resized.save(output, format='PNG')
processed_bytes = output.getvalue()
# Send binary data back
await websocket.send_bytes(processed_bytes)
print(f"Sent {len(processed_bytes)} bytes")
except WebSocketDisconnect:
print("Client disconnected")Run the server:
uvicorn image_server:app --reload
Client Implementation:
Install websockets if you haven't already:
pip install websockets
Then create the client:
# image_client.py
import asyncio
import websockets
async def send_image():
uri = "ws://localhost:8000/ws/images"
async with websockets.connect(uri) as websocket:
# Read image file
with open("photo.jpg", "rb") as f:
image_data = f.read()
# Send as binary
await websocket.send(image_data)
# Receive processed image
processed = await websocket.recv()
# Save result
with open("photo_resized.png", "wb") as f:
f.write(processed)
print("Image processed and saved!")
asyncio.run(send_image())Mixing Text and Binary
WebSockets support mixing text and binary messages in the same connection:
await websocket.send_text("Sending image...")
await websocket.send_bytes(image_data)
await websocket.send_json({"status": "complete"})WebSocket Authentication and Security
WebSockets need authentication just like HTTP endpoints. However, you can't send headers after the initial handshake, so authentication must happen during connection or via query parameters/cookies.
Authentication via Query Parameters
Server Implementation:
First, install the required dependencies:
pip install fastapi pyjwt uvicorn
Then create the authenticated server:
# auth_server.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect, HTTPException, status
import jwt
from datetime import datetime, timedelta, timezone
app = FastAPI()
# Secret key for JWT
SECRET_KEY = "your-secret-key"
ALGORITHM = "HS256"
def create_access_token(user_id: str) -> str:
"""Generate JWT token"""
payload = {
"user_id": user_id,
"exp": datetime.now(timezone.utc) + timedelta(hours=1)
}
return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
def verify_token(token: str) -> dict:
"""Verify JWT token and return payload"""
try:
payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
return payload
except jwt.ExpiredSignatureError:
raise HTTPException(status_code=401, detail="Token expired")
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid token")
@app.websocket("/ws/chat")
async def authenticated_chat(websocket: WebSocket, token: str):
"""
WebSocket endpoint with authentication.
Usage:
ws://localhost:8000/ws/chat?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
The token parameter is extracted from query string.
"""
# Verify token before accepting connection
try:
payload = verify_token(token)
user_id = payload["user_id"]
except HTTPException as e:
# Reject connection with close code
await websocket.close(code=status.WS_1008_POLICY_VIOLATION, reason=e.detail)
return
# Token valid - accept connection
await websocket.accept()
print(f"Authenticated user: {user_id}")
try:
while True:
data = await websocket.receive_text()
# Include user_id with message
response = {
"user_id": user_id,
"message": data,
"timestamp": datetime.now(timezone.utc).isoformat()
}
await websocket.send_json(response)
except WebSocketDisconnect:
print(f"User {user_id} disconnected")Run the server:
uvicorn auth_server:app --reload
Client Implementation:
Install the required dependencies:
pip install websockets requests
Then create the authenticated client:
# auth_client.py
import asyncio
import websockets
import requests
async def connect_with_auth():
# Get token from login endpoint
# Note: You'll need to implement a /login endpoint that returns a token
response = requests.post("http://localhost:8000/login", json={
"username": "alice",
"password": "secret"
})
token = response.json()["access_token"]
# Connect WebSocket with token in query parameter
uri = f"ws://localhost:8000/ws/chat?token={token}"
async with websockets.connect(uri) as websocket:
await websocket.send("Hello!")
response = await websocket.recv()
print(response) # Includes authenticated user_id
asyncio.run(connect_with_auth())Authentication Methods Comparison
| Method | Pros | Cons |
|---|---|---|
| Query Parameter | Simple, works everywhere | Token visible in logs/URLs |
| Cookie | Automatic, secure (HttpOnly) | CSRF risk, same-origin only |
| First Message | Flexible, not in URL | Race conditions, complex |
Rate Limiting WebSocket Connections
Prevent abuse by limiting how many WebSocket connections a user can open and how fast they can send messages.
import redis
import time
from collections import defaultdict
redis_client = redis.Redis(host='localhost', port=6379, decode_responses=True)
# In-memory rate limiter (use Redis for production)
message_timestamps = defaultdict(list)
async def check_connection_limit(user_id: str, max_connections: int = 5) -> bool:
"""
Check if user has exceeded concurrent connection limit.
Args:
user_id: User identifier
max_connections: Maximum concurrent connections per user
Returns:
True if allowed, False if limit exceeded
"""
key = f"ws_connections:{user_id}"
connection_count = redis_client.get(key)
if connection_count and int(connection_count) >= max_connections:
return False
return True
async def check_message_rate_limit(user_id: str, limit: int = 10, window: int = 60) -> bool:
"""
Rate limit messages per user.
Args:
user_id: User identifier
limit: Max messages per window
window: Time window in seconds
Returns:
True if allowed, False if rate limited
"""
current_time = time.time()
window_start = current_time - window
# Get user's message timestamps
timestamps = message_timestamps[user_id]
# Remove old timestamps
timestamps[:] = [ts for ts in timestamps if ts > window_start]
# Check limit
if len(timestamps) >= limit:
return False
# Add current timestamp
timestamps.append(current_time)
return True
@app.websocket("/ws/chat")
async def rate_limited_chat(websocket: WebSocket, token: str):
"""Chat endpoint with rate limiting"""
# Authenticate
try:
payload = verify_token(token)
user_id = payload["user_id"]
except HTTPException:
await websocket.close(code=1008, reason="Authentication failed")
return
# Check connection limit
if not await check_connection_limit(user_id, max_connections=5):
await websocket.close(code=1008, reason="Too many connections")
return
# Increment connection count
connection_key = f"ws_connections:{user_id}"
redis_client.incr(connection_key)
redis_client.expire(connection_key, 3600)
await websocket.accept()
try:
while True:
data = await websocket.receive_text()
# Rate limit messages
if not await check_message_rate_limit(user_id, limit=10, window=60):
await websocket.send_json({
"type": "error",
"message": "Rate limit exceeded. Max 10 messages per minute."
})
continue
# Process message
await websocket.send_json({
"type": "message",
"text": data,
"user_id": user_id
})
except WebSocketDisconnect:
# Decrement connection count
redis_client.decr(connection_key)
# Result:
# User can open max 5 WebSocket connections
# User can send max 10 messages per minute
# Exceeded limits get error message or connection closedSecurity Best Practices
- Always authenticate: Verify user identity before accepting connection
- Use WSS (WebSocket Secure): TLS encryption like HTTPS
- Validate origin: Check
Originheader to prevent CORS attacks - Rate limit: Limit connections per user and messages per second
- Message size limits: Reject messages larger than reasonable size (e.g., 64KB)
- Timeout idle connections: Close connections inactive for 15+ minutes
- Validate messages: Use Pydantic models to validate incoming JSON
- Monitor connections: Track active connections, detect abuse patterns
Broadcasting and Room Management
Most real-time apps need rooms/channels. Chat rooms, game lobbies and collaborative documents where users join specific rooms and only receive messages relevant to that room.
Room-Based Chat System
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from typing import Dict, List, Set
from collections import defaultdict
import json
app = FastAPI()
class RoomManager:
"""
Manage WebSocket rooms for group chat.
Features:
- Join/leave rooms
- Broadcast to specific room
- List users in room
- Private messages
"""
def __init__(self):
# room_id -> set of WebSocket connections
self.rooms: Dict[str, Set[WebSocket]] = defaultdict(set)
# websocket -> user_info
self.users: Dict[WebSocket, dict] = {}
async def join_room(self, websocket: WebSocket, room_id: str, username: str):
"""Add user to room"""
self.rooms[room_id].add(websocket)
self.users[websocket] = {
"username": username,
"room": room_id
}
# Notify room
await self.broadcast_to_room(room_id, {
"type": "user_joined",
"username": username,
"room": room_id,
"users_count": len(self.rooms[room_id])
}, exclude=websocket)
async def leave_room(self, websocket: WebSocket):
"""Remove user from room"""
if websocket not in self.users:
return
user_info = self.users[websocket]
room_id = user_info["room"]
username = user_info["username"]
# Remove from room
self.rooms[room_id].discard(websocket)
del self.users[websocket]
# Notify room
await self.broadcast_to_room(room_id, {
"type": "user_left",
"username": username,
"room": room_id,
"users_count": len(self.rooms[room_id])
})
async def broadcast_to_room(self, room_id: str, message: dict, exclude: WebSocket = None):
"""Send message to all users in room"""
if room_id not in self.rooms:
return
disconnected = []
for connection in self.rooms[room_id]:
if connection == exclude:
continue
try:
await connection.send_json(message)
except Exception:
disconnected.append(connection)
# Clean up disconnected clients
for conn in disconnected:
await self.leave_room(conn)
def get_room_users(self, room_id: str) -> List[str]:
"""Get list of usernames in room"""
if room_id not in self.rooms:
return []
return [
self.users[ws]["username"]
for ws in self.rooms[room_id]
if ws in self.users
]
manager = RoomManager()
@app.websocket("/ws/rooms")
async def room_websocket(websocket: WebSocket, username: str):
"""
Room-based chat WebSocket.
Messages format:
- Join: {"action": "join", "room": "general"}
- Send: {"action": "message", "text": "Hello!"}
- Leave: {"action": "leave"}
"""
await websocket.accept()
current_room = None
try:
while True:
# Receive message
data = await websocket.receive_text()
message = json.loads(data)
action = message.get("action")
if action == "join":
room_id = message.get("room", "general")
# Leave current room if in one
if current_room:
await manager.leave_room(websocket)
# Join new room
await manager.join_room(websocket, room_id, username)
current_room = room_id
# Send confirmation
await websocket.send_json({
"type": "joined",
"room": room_id,
"users": manager.get_room_users(room_id)
})
elif action == "message":
if not current_room:
await websocket.send_json({
"type": "error",
"message": "Not in a room. Join a room first."
})
continue
# Broadcast message to room
await manager.broadcast_to_room(current_room, {
"type": "message",
"username": username,
"text": message.get("text", ""),
"room": current_room
})
elif action == "leave":
if current_room:
await manager.leave_room(websocket)
current_room = None
await websocket.send_json({
"type": "left",
"message": "Left the room"
})
elif action == "list_users":
if current_room:
users = manager.get_room_users(current_room)
await websocket.send_json({
"type": "user_list",
"users": users,
"count": len(users)
})
except WebSocketDisconnect:
if current_room:
await manager.leave_room(websocket)
# Client example
async def chat_in_rooms():
uri = "ws://localhost:8000/ws/rooms?username=Alice"
async with websockets.connect(uri) as websocket:
# Join room
await websocket.send(json.dumps({
"action": "join",
"room": "python-devs"
}))
# Receive confirmation
response = json.loads(await websocket.recv())
print(f"Joined {response['room']}, users: {response['users']}")
# Send message to room
await websocket.send(json.dumps({
"action": "message",
"text": "Hello Python developers!"
}))
# Switch room
await websocket.send(json.dumps({
"action": "join",
"room": "javascript-devs"
}))
# Send to new room
await websocket.send(json.dumps({
"action": "message",
"text": "Hello JS developers!"
}))Result: Room-Based Chat
Scenario: 3 users in 2 different rooms
- Alice joins "python-devs": Gets notified, sees user list: [Alice]
- Bob joins "python-devs": Alice gets "user_joined" notification, user list: [Alice, Bob]
- Charlie joins "javascript-devs": Different room, Alice/Bob don't see notification
- Alice sends "Hello!": Only Bob receives (same room), Charlie doesn't (different room)
- Alice switches to "javascript-devs": Bob gets "user_left", Charlie gets "user_joined"
Presence Tracking (Who's Online)
import asyncio
from datetime import datetime, timezone
class PresenceManager:
"""Track online/offline status and last seen"""
def __init__(self):
self.online_users: Dict[str, dict] = {}
async def set_online(self, user_id: str, websocket: WebSocket):
"""Mark user as online"""
self.online_users[user_id] = {
"status": "online",
"websocket": websocket,
"connected_at": datetime.now(timezone.utc).isoformat(),
"last_activity": datetime.now(timezone.utc).isoformat()
}
# Broadcast presence update
await self.broadcast_presence_update(user_id, "online")
async def set_offline(self, user_id: str):
"""Mark user as offline"""
if user_id in self.online_users:
del self.online_users[user_id]
# Broadcast presence update
await self.broadcast_presence_update(user_id, "offline")
async def update_activity(self, user_id: str):
"""Update last activity timestamp"""
if user_id in self.online_users:
self.online_users[user_id]["last_activity"] = datetime.now(timezone.utc).isoformat()
def get_online_users(self) -> List[dict]:
"""Get list of online users"""
return [
{
"user_id": user_id,
"status": info["status"],
"connected_at": info["connected_at"]
}
for user_id, info in self.online_users.items()
]
async def broadcast_presence_update(self, user_id: str, status: str):
"""Notify all users about presence change"""
message = {
"type": "presence",
"user_id": user_id,
"status": status,
"timestamp": datetime.now(timezone.utc).isoformat()
}
for user_info in self.online_users.values():
try:
await user_info["websocket"].send_json(message)
except Exception:
pass
presence = PresenceManager()
@app.websocket("/ws/presence")
async def presence_websocket(websocket: WebSocket, user_id: str):
"""WebSocket with presence tracking"""
await websocket.accept()
# Mark user as online
await presence.set_online(user_id, websocket)
# Send current online users
await websocket.send_json({
"type": "online_users",
"users": presence.get_online_users()
})
try:
while True:
data = await websocket.receive_text()
# Update last activity
await presence.update_activity(user_id)
# Handle message...
except WebSocketDisconnect:
# Mark user as offline
await presence.set_offline(user_id)
# All connected clients receive:
# {"type": "presence", "user_id": "alice", "status": "online"}
# {"type": "presence", "user_id": "bob", "status": "online"}
# {"type": "presence", "user_id": "alice", "status": "offline"}Server-Sent Events (SSE): Simpler Alternative
Server-Sent Events (SSE) are a simpler alternative to WebSockets for one-way server-to-client communication. Perfect for notifications, live updates, and real-time dashboards where the client doesn't need to send data frequently.
SSE vs WebSockets
| Feature | Server-Sent Events | WebSockets |
|---|---|---|
| Direction | One-way (server → client) | Bidirectional (both ways) |
| Protocol | HTTP (reuses connections) | WebSocket (separate protocol) |
| Complexity | Simple (just HTTP streaming) | Complex (handshake, frames) |
| Browser Support | Built-in EventSource API | Built-in WebSocket API |
| Auto Reconnect | Automatic (3s default) | Manual implementation needed |
| Data Format | Text only (UTF-8) | Text or binary |
| Use Cases | Notifications, feeds, dashboards | Chat, games, collaboration |
Implementing SSE in FastAPI
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio
import json
import time
app = FastAPI()
async def event_generator():
"""
Generate server-sent events.
SSE format:
data: message
With event type:
event: notification
data: message
"""
counter = 0
while True:
# Simulate new data
counter += 1
# Format as SSE event
data = json.dumps({
"counter": counter,
"timestamp": time.time(),
"message": f"Update #{counter}"
})
# SSE format: "data: " prefix, double newline at end
yield f"data: {data}
"
# Send every 2 seconds
await asyncio.sleep(2)
@app.get("/sse/events")
async def sse_endpoint():
"""
Server-Sent Events endpoint.
Returns a stream of events to the client.
Client automatically reconnects if disconnected.
"""
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no" # Disable nginx buffering
}
)
# JavaScript client:
# const eventSource = new EventSource('http://localhost:8000/sse/events');
#
# eventSource.onmessage = (event) => {
# const data = JSON.parse(event.data);
# console.log('Received:', data);
# };
#
# eventSource.onerror = (error) => {
# console.error('SSE error:', error);
# // EventSource auto-reconnects after 3 seconds
# };
# Python client
import requests
def sse_client():
"""Receive server-sent events"""
response = requests.get(
'http://localhost:8000/sse/events',
stream=True
)
for line in response.iter_lines():
if line:
line = line.decode('utf-8')
if line.startswith('data: '):
data = json.loads(line[6:]) # Remove "data: " prefix
print(f"Received: {data}")Result: SSE Stream
# Client console output:
Received: {"counter": 1, "message": "Update #1"}
Received: {"counter": 2, "message": "Update #2"}
Received: {"counter": 3, "message": "Update #3"}
# (continues every 2 seconds)
# If connection drops:
# EventSource automatically reconnects
# Server resumes from where it left offSSE with Event Types and IDs
async def notification_generator():
"""
SSE with event types and IDs.
Event types allow different handlers:
- eventSource.addEventListener('notification', handler)
- eventSource.addEventListener('alert', handler)
Event IDs enable resuming from last received event.
"""
event_id = 0
while True:
event_id += 1
# Randomly choose event type
if event_id % 3 == 0:
event_type = "alert"
data = {"severity": "high", "message": "Important alert!"}
else:
event_type = "notification"
data = {"message": f"Notification #{event_id}"}
# SSE format with event type and ID
yield f"id: {event_id}
" # Event ID for resume
yield f"event: {event_type}
" # Event type
yield f"data: {json.dumps(data)}
" # Data (double newline at end)
await asyncio.sleep(2)
@app.get("/sse/notifications")
async def sse_notifications(last_event_id: str = None):
"""
SSE endpoint with resume support.
If client reconnects, they send Last-Event-ID header.
Server can resume from that point.
"""
# Use last_event_id to resume from specific event
# (implementation depends on your event storage)
return StreamingResponse(
notification_generator(),
media_type="text/event-stream"
)
# JavaScript client with event types:
# const eventSource = new EventSource('/sse/notifications');
#
# eventSource.addEventListener('notification', (event) => {
# const data = JSON.parse(event.data);
# console.log('Notification:', data);
# });
#
# eventSource.addEventListener('alert', (event) => {
# const data = JSON.parse(event.data);
# console.warn('ALERT:', data);
# });
#
# // On reconnect, browser sends Last-Event-ID header automatically
# // Server can resume from that eventWhen to Use SSE Instead of WebSockets
- One-way updates: Server pushes, client doesn't need to send much
- Simpler implementation: Just HTTP, no protocol upgrade complexity
- Auto-reconnect: Built-in reconnection logic (no code needed)
- Browser compatibility: Works with older browsers (polyfill available)
- HTTP/2 friendly: Reuses HTTP connections efficiently
- Examples: Live notifications, news feeds, stock tickers, progress updates
- Not for: Chat (bidirectional), gaming (low latency), file streaming (binary)
Scaling WebSockets Across Multiple Servers
Single-server WebSockets don't scale. When you have multiple API servers, a WebSocket connection to Server A can't receive messages broadcast from Server B. Solution: Redis pub/sub for cross-server messaging.
The Scaling Problem
Problem: Isolated WebSocket Connections
Scenario with 2 servers:
- Alice connects to Server A (WebSocket connection on Server A)
- Bob connects to Server B (WebSocket connection on Server B)
- Alice sends message → Server A broadcasts → Only reaches Server A connections
- Bob doesn't receive it! He's connected to Server B, which didn't see the message
- Problem: In-memory connection managers don't span servers
Solution: Redis Pub/Sub
Use Redis as a message bus. When Server A receives a message, it publishes to Redis. All servers (A, B, C...) subscribe to Redis and broadcast to their local WebSocket connections.
from fastapi import FastAPI, WebSocket
import redis.asyncio as redis
import json
import asyncio
import time
app = FastAPI()
# Redis connection for pub/sub
redis_client = None
class ScalableConnectionManager:
"""
WebSocket manager with Redis pub/sub for scaling.
Architecture:
- Each server has local WebSocket connections
- Messages published to Redis channel
- All servers subscribe to Redis
- Each server broadcasts to local connections
"""
def __init__(self):
self.active_connections: List[WebSocket] = []
self.redis_client = None
self.pubsub = None
async def connect(self, websocket: WebSocket):
"""Accept WebSocket connection"""
await websocket.accept()
self.active_connections.append(websocket)
def disconnect(self, websocket: WebSocket):
"""Remove WebSocket connection"""
self.active_connections.remove(websocket)
async def setup_redis(self):
"""Setup Redis connection and pub/sub"""
self.redis_client = await redis.from_url("redis://localhost:6379")
self.pubsub = self.redis_client.pubsub()
# Subscribe to broadcast channel
await self.pubsub.subscribe("chat_messages")
# Start listening in background
asyncio.create_task(self.listen_to_redis())
async def listen_to_redis(self):
"""
Listen to Redis pub/sub messages.
When message received from Redis:
1. Parse message
2. Broadcast to all local WebSocket connections
"""
async for message in self.pubsub.listen():
if message["type"] == "message":
# Parse message
data = json.loads(message["data"])
# Broadcast to all local connections
await self.broadcast_local(data)
async def broadcast_local(self, message: dict):
"""Broadcast to local WebSocket connections only"""
disconnected = []
for connection in self.active_connections:
try:
await connection.send_json(message)
except Exception:
disconnected.append(connection)
# Clean up
for conn in disconnected:
self.disconnect(conn)
async def broadcast_global(self, message: dict):
"""
Broadcast to ALL servers via Redis.
Steps:
1. Publish message to Redis channel
2. Redis delivers to all subscribed servers
3. Each server broadcasts to local connections
"""
# Publish to Redis
await self.redis_client.publish(
"chat_messages",
json.dumps(message)
)
manager = ScalableConnectionManager()
@app.on_event("startup")
async def startup():
"""Setup Redis on app startup"""
await manager.setup_redis()
@app.on_event("shutdown")
async def shutdown():
"""Cleanup Redis on shutdown"""
if manager.redis_client:
await manager.redis_client.close()
@app.websocket("/ws/chat")
async def scalable_chat(websocket: WebSocket):
"""
Scalable chat endpoint.
Works across multiple servers via Redis pub/sub.
"""
await manager.connect(websocket)
try:
while True:
# Receive message from client
data = await websocket.receive_text()
# Create message
message = {
"type": "message",
"text": data,
"timestamp": time.time()
}
# Broadcast to ALL servers (via Redis)
await manager.broadcast_global(message)
except WebSocketDisconnect:
manager.disconnect(websocket)
# Flow with 2 servers:
# 1. Alice (Server A) sends "Hello"
# 2. Server A publishes to Redis: {"text": "Hello"}
# 3. Redis notifies Server A and Server B
# 4. Server A broadcasts to local connections (Alice)
# 5. Server B broadcasts to local connections (Bob)
# 6. Both Alice and Bob receive "Hello"!Result: Cross-Server Communication
With Redis pub/sub:
- Server A: Alice and Charlie connected (2 WebSockets)
- Server B: Bob and Diana connected (2 WebSockets)
- Alice sends "Hello": Server A publishes to Redis
- Redis notifies both servers: A and B receive message
- Server A broadcasts: Alice and Charlie receive message
- Server B broadcasts: Bob and Diana receive message
- ✓ All 4 users receive message regardless of server!
Load Balancing WebSockets
# Nginx configuration for WebSocket load balancing
upstream websocket_servers {
# Use IP hash for sticky sessions
# Same client always routed to same server
ip_hash;
server 10.0.1.1:8000;
server 10.0.1.2:8000;
server 10.0.1.3:8000;
}
server {
listen 80;
server_name api.example.com;
location /ws/ {
# WebSocket proxy settings
proxy_pass http://websocket_servers;
proxy_http_version 1.1;
# Required for WebSocket upgrade
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Forward headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Timeouts
proxy_read_timeout 86400s; # 24 hours
proxy_send_timeout 86400s;
# Disable buffering
proxy_buffering off;
}
}
# Why ip_hash?
# - Same client connects to same server
# - Reduces Redis traffic (local broadcasts more common)
# - Better for presence tracking (user's connections on one server)
#
# Alternative: least_conn (connect to least busy server)
# Better distribution but more Redis trafficProduction Best Practices
Production WebSocket systems need heartbeats, connection limits, monitoring, graceful degradation, and proper error handling. Learn from production deployments.
Heartbeat / Ping-Pong
Detect dead connections with periodic ping/pong messages. Without heartbeats, you can't tell if a client crashed or if the network died silently.
import asyncio
from fastapi import WebSocket
HEARTBEAT_INTERVAL = 30 # Send ping every 30 seconds
HEARTBEAT_TIMEOUT = 60 # Close if no pong in 60 seconds
async def heartbeat_handler(websocket: WebSocket):
"""
Send periodic ping frames.
If client doesn't respond with pong within timeout, close connection.
"""
while True:
try:
# Send ping
await websocket.send_text('{"type": "ping"}')
# Wait for pong (with timeout)
try:
pong = await asyncio.wait_for(
websocket.receive_text(),
timeout=HEARTBEAT_TIMEOUT
)
# Check if it's a pong response
if '"type": "pong"' not in pong:
# Not a pong, put message back in queue
pass
except asyncio.TimeoutError:
# No pong received, connection is dead
print("Heartbeat timeout - closing connection")
await websocket.close(code=1000, reason="Heartbeat timeout")
break
# Wait before next ping
await asyncio.sleep(HEARTBEAT_INTERVAL)
except Exception as e:
print(f"Heartbeat error: {e}")
break
@app.websocket("/ws")
async def websocket_with_heartbeat(websocket: WebSocket):
"""WebSocket with heartbeat monitoring"""
await websocket.accept()
# Start heartbeat in background
heartbeat_task = asyncio.create_task(heartbeat_handler(websocket))
try:
while True:
data = await websocket.receive_text()
# Handle pong responses
message = json.loads(data)
if message.get("type") == "pong":
continue # Just acknowledgment, no processing
# Process regular message
await websocket.send_text(f"Echo: {data}")
except WebSocketDisconnect:
heartbeat_task.cancel()
print("Client disconnected")
# Client should respond to pings:
# websocket.onmessage = (event) => {
# const data = JSON.parse(event.data);
# if (data.type === 'ping') {
# websocket.send(JSON.stringify({type: 'pong'}));
# }
# };Connection Limits and Resource Management
from collections import defaultdict
# Global limits
MAX_CONNECTIONS_PER_USER = 5
MAX_TOTAL_CONNECTIONS = 10000
MAX_MESSAGE_SIZE = 64 * 1024 # 64 KB
# Track connections
connection_counts = defaultdict(int)
total_connections = 0
async def check_connection_limits(user_id: str) -> bool:
"""Check if user can open new connection"""
global total_connections
# Check total server limit
if total_connections >= MAX_TOTAL_CONNECTIONS:
return False
# Check per-user limit
if connection_counts[user_id] >= MAX_CONNECTIONS_PER_USER:
return False
return True
@app.websocket("/ws")
async def limited_websocket(websocket: WebSocket, user_id: str):
"""WebSocket with connection limits"""
global total_connections
# Check limits
if not await check_connection_limits(user_id):
await websocket.close(
code=1008,
reason="Connection limit exceeded"
)
return
# Accept connection
await websocket.accept()
# Track connection
connection_counts[user_id] += 1
total_connections += 1
try:
while True:
# Receive with size limit
data = await websocket.receive_text()
if len(data) > MAX_MESSAGE_SIZE:
await websocket.send_json({
"type": "error",
"message": "Message too large (max 64KB)"
})
continue
# Process message
await websocket.send_text(f"Echo: {data}")
except WebSocketDisconnect:
pass
finally:
# Cleanup
connection_counts[user_id] -= 1
total_connections -= 1
print(f"Total connections: {total_connections}")
# Monitoring endpoint
@app.get("/metrics/websockets")
async def websocket_metrics():
"""Get WebSocket connection metrics"""
return {
"total_connections": total_connections,
"max_connections": MAX_TOTAL_CONNECTIONS,
"usage_percent": (total_connections / MAX_TOTAL_CONNECTIONS) * 100,
"connections_by_user": dict(connection_counts)
}Graceful Shutdown
import signal
# Track active connections for graceful shutdown
active_websockets: Set[WebSocket] = set()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
"""WebSocket with graceful shutdown support"""
await websocket.accept()
active_websockets.add(websocket)
try:
while True:
data = await websocket.receive_text()
await websocket.send_text(f"Echo: {data}")
except WebSocketDisconnect:
pass
finally:
active_websockets.discard(websocket)
async def graceful_shutdown():
"""
Gracefully close all WebSocket connections.
Notify clients that server is shutting down,
give them time to reconnect to another server.
"""
print(f"Shutting down {len(active_websockets)} WebSocket connections...")
# Notify all clients
for ws in active_websockets:
try:
await ws.send_json({
"type": "server_shutdown",
"message": "Server restarting, please reconnect",
"reconnect_delay": 5 # Wait 5 seconds before reconnecting
})
# Close gracefully
await ws.close(code=1001, reason="Server going away")
except Exception as e:
print(f"Error closing WebSocket: {e}")
print("All WebSocket connections closed")
@app.on_event("shutdown")
async def shutdown_event():
"""Handle application shutdown"""
await graceful_shutdown()
# For deployment with zero downtime:
# 1. Deploy new server instances
# 2. Update load balancer to route new connections to new servers
# 3. Old servers finish handling existing connections
# 4. Gracefully shut down old servers after 5-10 minutesProduction Checklist
WebSocket Production Checklist
- ✓ Authentication: Verify user identity before accepting connection
- ✓ WSS (secure): Use TLS encryption (wss:// not ws://)
- ✓ Heartbeat: Ping/pong every 30-60 seconds to detect dead connections
- ✓ Connection limits: Max 5-10 per user, max 10k-100k per server
- ✓ Message size limits: Reject messages >64KB to prevent memory issues
- ✓ Rate limiting: Max 10-100 messages per second per connection
- ✓ Graceful shutdown: Notify clients before closing connections
- ✓ Redis pub/sub: For scaling across multiple servers
- ✓ Monitoring: Track active connections, message rate, errors
- ✓ Auto-reconnect: Client should reconnect with exponential backoff
- ✓ Fallback: Support SSE or long-polling if WebSockets fail
- ✓ Load balancing: Use ip_hash or sticky sessions
Real-World Examples
| Company | WebSocket Use Cases | Scale |
|---|---|---|
| Slack | Real-time messaging, presence, typing indicators | Millions of concurrent connections |
| Discord | Voice/video metadata, chat, game activity | Billions of messages per day |
| Trello | Collaborative board updates, card movements | Real-time sync across users |
| Figma | Multiplayer design editing, cursor positions | Sub-100ms latency updates |
| Coinbase | Live cryptocurrency price updates | 100k+ price updates per second |