Error Handling & Custom Exceptions

Master professional error handling with custom exceptions, exception hierarchies, context managers, and production-ready patterns.

Prerequisites

This lesson builds on the basic error handling from Python Foundations (try-except basics). You should be comfortable with classes and OOP concepts before proceeding.

Why Custom Exceptions?

While Python provides many built-in exceptions like ValueError and TypeError, custom exceptions allow you to create meaningful error types specific to your application domain. They make your code more maintainable, testable, and easier to debug.

Benefits

Domain-specific errors: Express business logic failures clearly
Better debugging: Meaningful exception names tell you what went wrong
Selective catching: Catch specific errors without catching everything
Additional context: Include custom attributes and messages

When to Use

Application-specific failures (e.g., PaymentFailedError)
Library/API development to provide clear error contracts
Validation errors with additional context
Workflow or state machine errors

Creating Custom Exception Classes

Basic Custom Exception

The simplest custom exception just inherits from Exception. This is sufficient when you only need a unique exception type.

# Define a basic custom exception
class InvalidAgeError(Exception):
    """Raised when age is invalid"""


# Using the custom exception
def set_age(age):
    if age < 0 or age > 150:
        raise InvalidAgeError(f"Age {age} is not valid")
    return age


# Test it
try:
    set_age(-5)
except InvalidAgeError as e:
    print(f"Error: {e}")
    # Error: Age -5 is not valid

Naming Convention: Exception class names should end with "Error" or "Exception" to make their purpose immediately clear.

Custom Exceptions with Attributes

Add custom attributes to provide additional context about the error. This makes debugging much easier and allows programmatic access to error details.

class InsufficientFundsError(Exception):
    """Raised when account has insufficient funds"""

    def __init__(self, balance, amount):
        self.balance = balance
        self.amount = amount
        self.shortage = amount - balance

        message = f"Insufficient funds: need ${amount}, have ${balance}"
        super().__init__(message)


# Using the exception
def withdraw(balance, amount):
    if amount > balance:
        raise InsufficientFundsError(balance, amount)
    return balance - amount

# Catching and using the exception
try:
    new_balance = withdraw(50, 100)
except InsufficientFundsError as e:
    print(f"Error: {e}")
    print(f"You're short by: ${e.shortage}")
    # Error: Insufficient funds: need $100, have $50
    # You're short by: $50

Adding Custom Methods

Custom exceptions can include methods to provide formatted output or additional functionality.

class ValidationError(Exception):
    """Raised when data validation fails"""

    def __init__(self, field, value, reason):
        self.field = field
        self.value = value
        self.reason = reason
        super().__init__(f"Validation failed for {field}")

    def get_details(self):
        """Return formatted error details"""
        return {
            'field': self.field,
            'invalid_value': self.value,
            'reason': self.reason
        }

    def user_friendly_message(self):
        """Return a message suitable for end users"""
        return f"{self.field.title()}: {self.reason}"

# Using the exception with methods
def validate_email(email):
    if '@' not in email:
        raise ValidationError('email', email, 'must contain @ symbol')
    return True


try:
    validate_email('invalid-email')
except ValidationError as e:
    print(e.user_friendly_message())
    print(e.get_details())
    # Email: must contain @ symbol
    # {'field': 'email', 'invalid_value': 'invalid-email', 'reason': 'must contain @ symbol'}

Exception Hierarchies

Creating a hierarchy of exceptions allows you to catch errors at different levels of specificity. This is especially useful in libraries and large applications.

Building an Exception Hierarchy

# Base exception for your application
class AppError(Exception):
    """Base exception for all application errors"""


# Database-related errors
class DatabaseError(AppError):
    """Base exception for database errors"""

class ConnectionError(DatabaseError):
    """Database connection failed"""

class QueryError(DatabaseError):
    """Database query failed"""


# Authentication-related errors
class AuthError(AppError):
    """Base exception for authentication errors"""
    pass

class InvalidCredentialsError(AuthError):
    """Username or password incorrect"""
    pass

class TokenExpiredError(AuthError):
    """Authentication token has expired"""
    pass

The hierarchy structure:

Exception (built-in)
└── AppError (your base)
    ├── DatabaseError
    │   ├── ConnectionError
    │   └── QueryError
    └── AuthError
        ├── InvalidCredentialsError
        └── TokenExpiredError

Catching Exceptions by Hierarchy

With a hierarchy, you can catch specific exceptions or broad categories depending on your needs.

def connect_to_database():
    raise ConnectionError("Could not connect to database server")

def authenticate_user(username, password):
    raise InvalidCredentialsError("Invalid password")


# Catch specific exception
try:
    connect_to_database()
except ConnectionError as e:
    print(f"Connection issue: {e}")
    # Connection issue: Could not connect to database server

# Catch all database errors
try:
    connect_to_database()
except DatabaseError as e:
    print(f"Database error: {e}")
    # Database error: Could not connect to database server


# Catch all application errors
try:
    authenticate_user('john', 'wrong_password')
except AppError as e:
    print(f"Application error: {e}")
    # Application error: Invalid password

Real-World Example: API Client

class APIError(Exception):
    """Base exception for API errors"""
    def __init__(self, message, status_code=None):
        super().__init__(message)
        self.status_code = status_code

class ClientError(APIError):
    """4xx client errors"""

class NotFoundError(ClientError):
    """404 Not Found"""
    def __init__(self, resource):
        super().__init__(f"Resource not found: {resource}", 404)
        self.resource = resource

class UnauthorizedError(ClientError):
    """401 Unauthorized"""
    def __init__(self):
        super().__init__("Authentication required", 401)

class ServerError(APIError):
    """5xx server errors"""
    pass

def make_api_request(endpoint, authenticated=False):
    if endpoint == "/user/123":
        raise NotFoundError("/user/123")
    if not authenticated:
        raise UnauthorizedError()


# Different handling strategies
try:
    make_api_request("/user/123")
except NotFoundError as e:
    print(f"Specific: {e.resource} does not exist")
except ClientError as e:
    print(f"Client error ({e.status_code}): {e}")
except APIError as e:
    print(f"General API error: {e}")

# Specific: /user/123 does not exist

Context Managers for Cleanup

Context managers ensure that resources are properly cleaned up, even when exceptions occur. The with statement guarantees cleanup code runs.

Review: The `with` Statement

# Without context manager - risky!
file = open('data.txt', 'r')
try:
    content = file.read()
    # If an exception occurs here, file might not close
    process(content)
finally:
    file.close()


# With context manager - guaranteed cleanup
with open('data.txt', 'r') as file:
    content = file.read()
    process(content)
    # File automatically closes, even if exception occurs

Creating Custom Context Managers (Class-Based)

Implement __enter__ and __exit__ methods to create a context manager.

class DatabaseConnection:
    """Context manager for database connections"""

    def __init__(self, host, database):
        self.host = host
        self.database = database
        self.connection = None

    def __enter__(self):
        """Called when entering 'with' block"""
        print(f"Opening connection to {self.database}")
        # In real code: self.connection = connect(self.host, self.database)
        self.connection = f"Connected to {self.database}"
        return self.connection

    def __exit__(self, exc_type, exc_value, traceback):
        """Called when exiting 'with' block"""
        print(f"Closing connection to {self.database}")
        # In real code: self.connection.close()

        # Return False to propagate exceptions
        # Return True to suppress exceptions
        return False

# Using the context manager
with DatabaseConnection('localhost', 'mydb') as conn:
    print(f"Using: {conn}")
    # Do work with connection

# Output:
# Opening connection to mydb
# Using: Connected to mydb
# Closing connection to mydb

Creating Context Managers with `@contextmanager`

The @contextmanager decorator provides a simpler way to create context managers using generator functions.

from contextlib import contextmanager

@contextmanager
def temporary_setting(name, value):
    """Temporarily change a setting, then restore it"""

    # Setup (before yield = __enter__)
    old_value = get_setting(name)
    set_setting(name, value)
    print(f"Setting {name} = {value}")

    try:
        yield  # This is where the 'with' block executes
    finally:
        # Cleanup (after yield = __exit__)
        set_setting(name, old_value)
        print(f"Restored {name} = {old_value}")


# Mock functions for example
settings = {'debug': False}
def get_setting(name): return settings.get(name)
def set_setting(name, value): settings[name] = value

# Using the decorator-based context manager
with temporary_setting('debug', True):
    print(f"Debug is: {get_setting('debug')}")
    # Do work with debug enabled

print(f"After context: {get_setting('debug')}")

# Output:
# Setting debug = True
# Debug is: True
# Restored debug = False
# After context: False

Exception Handling in Context Managers

Context managers can handle exceptions that occur within the with block.

class TransactionManager:
    """Manages database transactions with rollback on error"""

    def __enter__(self):
        print("BEGIN TRANSACTION")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        if exc_type is None:
            # No exception occurred
            print("COMMIT TRANSACTION")
        else:
            # Exception occurred
            print(f"ROLLBACK TRANSACTION (error: {exc_value})")
            # Return True to suppress the exception
            # Return False to let it propagate
            return False  # Let exception propagate

# Successful transaction
with TransactionManager():
    print("Inserting data...")
    print("Updating records...")

# Output:
# BEGIN TRANSACTION
# Inserting data...
# Updating records...
# COMMIT TRANSACTION


# Failed transaction
try:
    with TransactionManager():
        print("Inserting data...")
        raise ValueError("Something went wrong!")
except ValueError:
    print("Exception handled outside context")

# Output:
# BEGIN TRANSACTION
# Inserting data...
# ROLLBACK TRANSACTION (error: Something went wrong!)
# Exception handled outside context

Best Practices for Error Handling

Do's and Don'ts

DO

Be specific: Catch specific exceptions, not broad ones
except ValueError as e:
Provide context: Include helpful error messages
raise ValueError(f"Age must be 0-150, got {age}")
Chain exceptions: Preserve original error context
raise NewError("message") from original_error
Document exceptions: Note which exceptions functions raise
"""Raises: ValueError if age invalid"""

DON'T

Catch everything: Don't use bare except:
except: # ❌ Too broad!
Swallow exceptions: Don't catch and ignore
except Exception: pass # ❌ Silent failure
Catch too much: Only catch what you can handle
except Exception as e: # ❌ Too broad
Lose information: Don't throw away the original error
raise NewError() # ❌ Lost original context

Exception Chaining: Preserving Context

Use raise ... from ... to preserve the original exception while raising a new one.

def load_config(filename):
    try:
        with open(filename) as f:
            return json.load(f)
    except FileNotFoundError as e:
        # Chain exceptions to preserve context
        raise ConfigurationError(
            f"Config file {filename} not found"
        ) from e
    except json.JSONDecodeError as e:
        raise ConfigurationError(
            f"Invalid JSON in {filename}"
        ) from e


class ConfigurationError(Exception):
    pass

# The chained exception shows both errors
try:
    import json
    load_config('missing.json')
except ConfigurationError as e:
    print(f"Error: {e}")
    print(f"Original cause: {e.__cause__}")

# Error: Config file missing.json not found
# Original cause: [Errno 2] No such file or directory: 'missing.json'

Fail Fast vs. Defensive Programming

Fail Fast

Validate inputs and raise exceptions immediately when something is wrong.

def divide(a, b):
    if b == 0:
        raise ValueError("Cannot divide by zero")

    return a / b

EAFP (Easier to Ask Forgiveness)

Try the operation and handle exceptions if they occur. Pythonic approach.

def get_value(data, key):
    try:
        return data[key]
    except KeyError:
        return None

Practical Examples

Example 1: Database Connection Manager

from contextlib import contextmanager

class DatabaseError(Exception):
    """Base exception for database operations"""
    pass

class ConnectionError(DatabaseError):
    """Failed to connect to database"""
    pass

class QueryError(DatabaseError):
    """Failed to execute query"""
    def __init__(self, query, original_error):
        self.query = query
        self.original_error = original_error
        super().__init__(f"Query failed: {query}")


@contextmanager
def database_session(connection_string):
    """Context manager for database sessions"""
    connection = None
    try:
        # Simulate connection
        print(f"Connecting to database...")
        connection = f"DB Connection: {connection_string}"
        yield connection
    except Exception as e:
        print(f"Rolling back transaction due to error")
        raise
    else:
        print(f"Committing transaction")
    finally:
        if connection:
            print(f"Closing connection")
            connection = None

# Using the database session manager
try:
    with database_session("localhost:5432/mydb") as db:
        print(f"Using: {db}")
        print("Executing queries...")
        # Simulate work
except DatabaseError as e:
    print(f"Database error: {e}")

# Output:
# Connecting to database...
# Using: DB Connection: localhost:5432/mydb
# Executing queries...
# Committing transaction
# Closing connection

Example 2: API Client with Custom Exceptions

class APIClient:
    """HTTP API client with comprehensive error handling"""

    def __init__(self, base_url, api_key):
        self.base_url = base_url
        self.api_key = api_key

    def get_user(self, user_id):
        """Fetch user by ID"""
        try:
            # Simulate API call
            if user_id < 1:
                raise ValueError("Invalid user ID")

            if user_id == 999:
                # Simulate 404
                raise NotFoundError(f"user/{user_id}")

            # Simulate success
            return {"id": user_id, "name": "John Doe"}

        except ValueError as e:
            raise ClientError(f"Invalid request: {e}", 400) from e

    def create_user(self, user_data):
        """Create a new user"""
        required_fields = ['name', 'email']

        # Validate input
        for field in required_fields:
            if field not in user_data:
                raise ValidationError(
                    field,
                    None,
                    f"{field} is required"
                )

        return {"id": 123, **user_data}

# Using the API client with proper error handling
client = APIClient("https://api.example.com", "secret-key")

# Handle specific errors
try:
    user = client.get_user(999)
except NotFoundError as e:
    print(f"User not found: {e.resource}")
except ClientError as e:
    print(f"Client error ({e.status_code}): {e}")
except APIError as e:
    print(f"API error: {e}")

# Output:
# User not found: user/999


# Validation errors
try:
    client.create_user({"name": "Alice"})
except ValidationError as e:
    print(e.user_friendly_message())
    # Email: email is required

Example 3: File Processor with Cleanup

class FileProcessingError(Exception):
    """Errors during file processing"""
    def __init__(self, filename, reason):
        self.filename = filename
        self.reason = reason
        super().__init__(f"Failed to process {filename}: {reason}")


class FileProcessor:
    """Process files with automatic cleanup"""

    def __init__(self, input_file, output_file):
        self.input_file = input_file
        self.output_file = output_file
        self.input_handle = None
        self.output_handle = None

    def __enter__(self):
        try:
            # Open files
            print(f"Opening {self.input_file} for reading")
            # In real code: self.input_handle = open(self.input_file, 'r')
            self.input_handle = True  # Simulated

            print(f"Opening {self.output_file} for writing")
            # In real code: self.output_handle = open(self.output_file, 'w')
            self.output_handle = True  # Simulated

            return self
        except IOError as e:
            raise FileProcessingError(
                self.input_file,
                f"Cannot open file: {e}"
            ) from e

    def __exit__(self, exc_type, exc_value, traceback):
        # Always close files, even if exception occurred
        if self.input_handle:
            print(f"Closing {self.input_file}")
            # self.input_handle.close()

        if self.output_handle:
            if exc_type is not None:
                print(f"Error occurred, removing {self.output_file}")
                # Remove partial output file
            print(f"Closing {self.output_file}")
            # self.output_handle.close()

        return False  # Propagate exceptions

    def process(self):
        """Process the file"""
        print("Processing file...")
        # In real code: read, transform, write
        return "Processed successfully"

# Using the file processor
try:
    with FileProcessor('input.txt', 'output.txt') as processor:
        result = processor.process()
        print(result)
except FileProcessingError as e:
    print(f"Processing failed: {e.reason}")

# Output:
# Opening input.txt for reading
# Opening output.txt for writing
# Processing file...
# Processed successfully
# Closing input.txt
# Closing output.txt

Key Takeaways

Custom exceptions make your code more maintainable by expressing domain-specific failures clearly
Exception hierarchies allow you to catch errors at different levels of specificity
Add custom attributes to exceptions to provide additional context for debugging
Context managers guarantee cleanup code runs, even when exceptions occur
Use @contextmanager decorator for simpler context manager creation
Chain exceptions with raise ... from ... to preserve error context
Be specific when catching exceptions - avoid broad catches like except Exception
Never swallow exceptions silently - always log or handle them appropriately
Document exceptions in docstrings so users know what to expect

What's Next?

You've mastered custom exceptions! Now let's learn powerful text processing with regular expressions.

Regex Patterns - Match and extract text patterns efficiently
Validation - Validate emails, URLs, and user input
Text Processing - Parse and transform text data with regex

Back to "Logging"

Python IntermediateLesson 13

Next: "Regular Expressions (Regex)"