Standard Library Essentials

Master the powerful tools that come built-in with Python

Introduction

Python's standard library is often called "batteries included", it comes with powerful modules for common tasks that you'll use in almost every project. From handling dates and times to working with JSON data, managing files, and using specialized data structures, these built-in tools save you from reinventing the wheel. Learning the standard library makes you immediately more productive and prepares you for real-world Python development. No installation required, these tools are ready to use right now!

The datetime Module

The datetime module provides classes for working with dates and times. It's essential for scheduling, logging, data analysis, and any application that needs to track when things happen.

Getting Current Date and Time

from datetime import datetime, date, time

# Get current date and time
now = datetime.now()
print(now)  # 2024-01-15 14:30:45.123456

# Get just the date
today = date.today()
print(today)  # 2024-01-15

# Access individual components
print(now.year)    # 2024
print(now.month)   # 1
print(now.day)     # 15
print(now.hour)    # 14
print(now.minute)  # 30
print(now.second)  # 45

# Get day of week (0 = Monday, 6 = Sunday)
print(today.weekday())  # 0 (if it's Monday)

# Get day name
print(today.strftime("%A"))  # "Monday"

Creating Specific Dates and Times

from datetime import datetime, date, time

# Create specific date
birthday = date(1990, 5, 15)  # May 15, 1990
print(birthday)  # 1990-05-15

# Create specific datetime
meeting = datetime(2024, 12, 25, 14, 30)  # Dec 25, 2024 at 2:30 PM
print(meeting)  # 2024-12-25 14:30:00

# Create specific time
alarm = time(7, 30, 0)  # 7:30:00 AM
print(alarm)  # 07:30:00

# Create time with microseconds
precise_time = time(12, 30, 45, 123456)
print(precise_time)  # 12:30:45.123456

Formatting Dates and Times

Use strftime() (string format time) to convert dates to readable strings.

from datetime import datetime

now = datetime.now()

# Common formats
print(now.strftime("%Y-%m-%d"))           # "2024-01-15"
print(now.strftime("%m/%d/%Y"))           # "01/15/2024"
print(now.strftime("%B %d, %Y"))          # "January 15, 2024"
print(now.strftime("%A, %B %d, %Y"))      # "Monday, January 15, 2024"

# Time formats
print(now.strftime("%H:%M:%S"))           # "14:30:45" (24-hour)
print(now.strftime("%I:%M %p"))           # "02:30 PM" (12-hour)

# Combined
print(now.strftime("%Y-%m-%d %H:%M:%S"))  # "2024-01-15 14:30:45"
print(now.strftime("%B %d, %Y at %I:%M %p"))
# "January 15, 2024 at 02:30 PM"

# Common format codes:
# %Y - Year (4 digits)      %m - Month (01-12)
# %d - Day (01-31)          %H - Hour 24h (00-23)
# %I - Hour 12h (01-12)     %M - Minute (00-59)
# %S - Second (00-59)       %p - AM/PM
# %A - Weekday full         %B - Month full
# %a - Weekday short        %b - Month short

Parsing Date Strings

Use strptime() (string parse time) to convert strings to datetime objects.

from datetime import datetime

# Parse different date formats
date_str1 = "2024-01-15"
dt1 = datetime.strptime(date_str1, "%Y-%m-%d")
print(dt1)  # 2024-01-15 00:00:00

date_str2 = "01/15/2024"
dt2 = datetime.strptime(date_str2, "%m/%d/%Y")
print(dt2)  # 2024-01-15 00:00:00

date_str3 = "January 15, 2024"
dt3 = datetime.strptime(date_str3, "%B %d, %Y")
print(dt3)  # 2024-01-15 00:00:00

# Parse datetime with time
datetime_str = "2024-01-15 14:30:45"
dt4 = datetime.strptime(datetime_str, "%Y-%m-%d %H:%M:%S")
print(dt4)  # 2024-01-15 14:30:45

# Handle different formats
formats = ["%Y-%m-%d", "%m/%d/%Y", "%d-%m-%Y"]
date_string = "15-01-2024"

for fmt in formats:
    try:
        parsed = datetime.strptime(date_string, fmt)
        print(f"Success: {parsed}")
        break
    except ValueError:
        continue

Date Arithmetic

Use timedelta to perform arithmetic with dates.

from datetime import datetime, timedelta, date

# Create timedelta
one_day = timedelta(days=1)
one_week = timedelta(weeks=1)
one_hour = timedelta(hours=1)
mixed = timedelta(days=7, hours=2, minutes=30)

# Add/subtract time
today = date.today()
tomorrow = today + timedelta(days=1)
yesterday = today - timedelta(days=1)
next_week = today + timedelta(weeks=1)

print(f"Today: {today}")
print(f"Tomorrow: {tomorrow}")
print(f"Yesterday: {yesterday}")
print(f"Next week: {next_week}")

# Calculate age
birthday = date(1990, 5, 15)
today = date.today()
age_delta = today - birthday
age_years = age_delta.days // 365
print(f"Age: approximately {age_years} years")

# Calculate days until event
event_date = date(2024, 12, 25)
today = date.today()
days_until = (event_date - today).days
print(f"Days until Christmas: {days_until}")

Practical datetime Examples

from datetime import datetime, timedelta

# Calculate project deadline
start_date = datetime(2024, 1, 15)
duration_days = 30
deadline = start_date + timedelta(days=duration_days)
print(f"Project deadline: {deadline.strftime('%B %d, %Y')}")
# "Project deadline: February 14, 2024"

# Check if date is weekend
def is_weekend(date):
    return date.weekday() >= 5  # 5 = Saturday, 6 = Sunday

today = datetime.now().date()
if is_weekend(today):
    print("It's the weekend!")
else:
    print("It's a weekday")

# Calculate business days between dates
def business_days_between(start, end):
    days = 0
    current = start
    while current <= end:
        if current.weekday() < 5:  # Monday-Friday
            days += 1
        current += timedelta(days=1)
    return days

start = date(2024, 1, 15)  # Monday
end = date(2024, 1, 19)    # Friday
print(f"Business days: {business_days_between(start, end)}")
# Business days: 5

# Format timestamps for logs
def log_message(message):
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    return f"[{timestamp}] {message}"

print(log_message("Application started"))
# "[2024-01-15 14:30:45] Application started"

# Calculate time remaining
deadline = datetime(2024, 12, 31, 23, 59, 59)
now = datetime.now()
remaining = deadline - now
print(f"Time remaining: {remaining.days} days, {remaining.seconds // 3600} hours")

Timezone Note: The examples above use naive datetime objects (no timezone info). For production apps with multiple timezones, use the pytz or zoneinfomodules for timezone-aware datetimes.

The json Module

JSON (JavaScript Object Notation) is the most common data format for APIs and configuration files. The json module makes it easy to convert between Python objects and JSON strings.

Converting Python to JSON (Serialization)

Use json.dumps() (dump string) to convert Python objects to JSON strings.

import json

# Python dictionary to JSON
person = {
    "name": "Alice",
    "age": 30,
    "city": "NYC",
    "is_student": False,
    "grades": [85, 90, 92]
}

json_string = json.dumps(person)
print(json_string)
# {"name": "Alice", "age": 30, "city": "NYC", "is_student": false, "grades": [85, 90, 92]}

# Pretty print with indentation
json_pretty = json.dumps(person, indent=4)
print(json_pretty)
# {
#     "name": "Alice",
#     "age": 30,
#     "city": "NYC",
#     "is_student": false,
#     "grades": [85, 90, 92]
# }

# Convert list to JSON
numbers = [1, 2, 3, 4, 5]
json_list = json.dumps(numbers)
print(json_list)  # [1, 2, 3, 4, 5]

# Nested structures
data = {
    "users": [
        {"id": 1, "name": "Alice"},
        {"id": 2, "name": "Bob"}
    ],
    "total": 2
}
json_nested = json.dumps(data, indent=2)
print(json_nested)

Converting JSON to Python (Deserialization)

Use json.loads() (load string) to convert JSON strings to Python objects.

import json

# JSON string to Python
json_string = '{"name": "Alice", "age": 30, "city": "NYC"}'
person = json.loads(json_string)
print(person)  # {"name": "Alice", "age": 30, "city": "NYC"}
print(type(person))  # <class 'dict'>

# Access the data
print(person["name"])  # "Alice"
print(person["age"])   # 30

# JSON array to Python list
json_array = '[1, 2, 3, 4, 5]'
numbers = json.loads(json_array)
print(numbers)  # [1, 2, 3, 4, 5]
print(type(numbers))  # <class 'list'>

# Complex nested structure
json_data = '''
{
    "users": [
        {"id": 1, "name": "Alice", "active": true},
        {"id": 2, "name": "Bob", "active": false}
    ],
    "total": 2,
    "page": 1
}
'''
data = json.loads(json_data)
print(data["users"][0]["name"])  # "Alice"
print(len(data["users"]))        # 2

Reading and Writing JSON Files

import json

# Write Python object to JSON file
data = {
    "name": "Alice",
    "age": 30,
    "hobbies": ["reading", "coding", "hiking"]
}

with open("person.json", "w") as file:
    json.dump(data, file, indent=4)  # dump (no 's') for files
# Creates person.json with formatted content

# Read JSON file to Python object
with open("person.json", "r") as file:
    loaded_data = json.load(file)  # load (no 's') for files

print(loaded_data["name"])  # "Alice"
print(loaded_data["hobbies"])  # ["reading", "coding", "hiking"]

# Append to existing JSON file (read, modify, write)
with open("person.json", "r") as file:
    data = json.load(file)

data["city"] = "NYC"  # Add new field
data["age"] = 31      # Update field

with open("person.json", "w") as file:
    json.dump(data, file, indent=4)

Practical JSON Examples

import json

# Configuration file
config = {
    "database": {
        "host": "localhost",
        "port": 5432,
        "name": "myapp"
    },
    "debug": True,
    "max_connections": 100
}

# Save configuration
with open("config.json", "w") as f:
    json.dump(config, f, indent=4)

# Load configuration
with open("config.json", "r") as f:
    loaded_config = json.load(f)
    db_host = loaded_config["database"]["host"]
    print(f"Database host: {db_host}")

# API response simulation
api_response = {
    "status": "success",
    "data": {
        "users": [
            {"id": 1, "username": "alice", "email": "alice@example.com"},
            {"id": 2, "username": "bob", "email": "bob@example.com"}
        ]
    },
    "timestamp": "2024-01-15T14:30:00Z"
}

# Convert to JSON string (as API would return)
json_response = json.dumps(api_response)

# Parse JSON response (as client would receive)
parsed = json.loads(json_response)
for user in parsed["data"]["users"]:
    print(f"{user['username']}: {user['email']}")
# alice: alice@example.com
# bob: bob@example.com

# Handle JSON errors
invalid_json = '{"name": "Alice", "age": }'  # Missing value
try:
    data = json.loads(invalid_json)
except json.JSONDecodeError as e:
    print(f"Invalid JSON: {e}")
    # Invalid JSON: Expecting value: line 1 column 25 (char 24)

JSON Type Mapping: Python's dict ↔ JSON object, list/tuple ↔ JSON array, str ↔ JSON string, int/float ↔ JSON number, True/False ↔ JSON true/false, None ↔ JSON null

The collections Module

The collections module provides specialized container data types that extend Python's built-in containers (dict, list, set, tuple) with additional functionality.

Counter - Count Occurrences

Counter is a dict subclass for counting hashable objects. Perfect for frequency analysis.

from collections import Counter

# Count items in a list
fruits = ["apple", "banana", "apple", "cherry", "banana", "apple"]
counter = Counter(fruits)
print(counter)
# Counter({"apple": 3, "banana": 2, "cherry": 1})

# Access counts
print(counter["apple"])   # 3
print(counter["cherry"])  # 1
print(counter["grape"])   # 0 (no KeyError!)

# Count characters in a string
text = "hello world"
char_count = Counter(text)
print(char_count)
# Counter({"l": 3, "o": 2, "h": 1, "e": 1, " ": 1, "w": 1, "r": 1, "d": 1})

# Most common elements
print(counter.most_common(2))  # [("apple", 3), ("banana", 2)]

# Count words in text
sentence = "the quick brown fox jumps over the lazy dog"
words = sentence.split()
word_count = Counter(words)
print(word_count.most_common(3))
# [("the", 2), ("quick", 1), ("brown", 1)]

Counter Operations

from collections import Counter

# Create Counter from different sources
c1 = Counter([1, 2, 2, 3, 3, 3])
c2 = Counter({"a": 3, "b": 2})
c3 = Counter(a=3, b=2)  # keyword arguments

# Arithmetic operations
c1 = Counter(a=3, b=1)
c2 = Counter(a=1, b=2)

# Addition
combined = c1 + c2
print(combined)  # Counter({"a": 4, "b": 3})

# Subtraction (keeps positive counts only)
diff = c1 - c2
print(diff)  # Counter({"a": 2})

# Intersection (minimum)
intersection = c1 & c2
print(intersection)  # Counter({"a": 1, "b": 1})

# Union (maximum)
union = c1 | c2
print(union)  # Counter({"a": 3, "b": 2})

# Update (add counts)
c1 = Counter(a=3, b=1)
c1.update({"a": 1, "b": 2, "c": 3})
print(c1)  # Counter({"a": 4, "b": 3, "c": 3})

# Get total count
total = sum(c1.values())
print(f"Total items: {total}")  # 10

Working with Files and Paths: os and pathlib

The os module provides operating system functionality, while pathlib offers an object-oriented approach to file paths. Both are essential for file and directory operations.

Basic os Module Operations

import os

# Get current working directory
cwd = os.getcwd()
print(f"Current directory: {cwd}")
# /home/user/projects

# Change directory
os.chdir("/tmp")
print(f"New directory: {os.getcwd()}")
# /tmp

# List files in directory
files = os.listdir(".")
print(files)
# ["file1.txt", "file2.py", "folder1"]

# Check if path exists
print(os.path.exists("file1.txt"))  # True
print(os.path.exists("nonexistent"))  # False

# Check if it's a file or directory
print(os.path.isfile("file1.txt"))  # True
print(os.path.isdir("folder1"))     # True

# Get file size (in bytes)
size = os.path.getsize("file1.txt")
print(f"Size: {size} bytes")

# Create directory
os.mkdir("new_folder")  # Single directory
os.makedirs("parent/child/grandchild")  # Nested directories

# Remove directory
os.rmdir("new_folder")  # Empty directory only
# Use shutil.rmtree() for non-empty directories

Working with Paths (os.path)

import os

# Join paths (handles OS differences)
path = os.path.join("folder", "subfolder", "file.txt")
print(path)
# Linux/Mac: folder/subfolder/file.txt
# Windows: folder\subfolder\file.txt

# Split path into directory and filename
full_path = "/home/user/documents/report.pdf"
directory, filename = os.path.split(full_path)
print(f"Directory: {directory}")  # /home/user/documents
print(f"Filename: {filename}")    # report.pdf

# Get filename without path
basename = os.path.basename(full_path)
print(basename)  # report.pdf

# Get directory without filename
dirname = os.path.dirname(full_path)
print(dirname)  # /home/user/documents

# Split filename and extension
name, ext = os.path.splitext("report.pdf")
print(f"Name: {name}")  # report
print(f"Extension: {ext}")  # .pdf

# Get absolute path
rel_path = "data/file.txt"
abs_path = os.path.abspath(rel_path)
print(abs_path)  # /home/user/projects/data/file.txt

# Check if path is absolute
print(os.path.isabs("/home/user"))  # True
print(os.path.isabs("data/file.txt"))  # False

Modern Path Handling with pathlib

pathlib provides an object-oriented interface that's more intuitive and Pythonic than os.path.

from pathlib import Path

# Create Path object
path = Path("folder/subfolder/file.txt")
print(path)  # folder/subfolder/file.txt

# Current directory
cwd = Path.cwd()
print(cwd)  # /home/user/projects

# Home directory
home = Path.home()
print(home)  # /home/user

# Join paths with / operator
data_dir = Path("data")
file_path = data_dir / "reports" / "2024" / "january.csv"
print(file_path)  # data/reports/2024/january.csv

# Check existence
print(file_path.exists())  # False (if doesn't exist)
print(file_path.is_file())  # False
print(file_path.is_dir())   # False

# Get parts of path
full_path = Path("/home/user/documents/report.pdf")
print(full_path.parent)  # /home/user/documents
print(full_path.name)    # report.pdf
print(full_path.stem)    # report
print(full_path.suffix)  # .pdf

# Get all parent directories
print(full_path.parents[0])  # /home/user/documents
print(full_path.parents[1])  # /home/user
print(full_path.parents[2])  # /home

File Operations with pathlib

from pathlib import Path

# Read file
file_path = Path("data.txt")
if file_path.exists():
    content = file_path.read_text()
    print(content)

# Write file
file_path.write_text("Hello, World!")

# Read/write bytes
binary_data = file_path.read_bytes()
file_path.write_bytes(b"Binary content")

# Create directory
new_dir = Path("new_folder")
new_dir.mkdir(exist_ok=True)  # No error if exists

# Create nested directories
nested = Path("parent/child/grandchild")
nested.mkdir(parents=True, exist_ok=True)

# List all files in directory
data_dir = Path("data")
for file in data_dir.iterdir():
    print(file)

# Find files by pattern (glob)
# Find all .txt files
txt_files = list(data_dir.glob("*.txt"))
print(txt_files)

# Find all .py files recursively
py_files = list(data_dir.rglob("*.py"))
print(py_files)

# Get file stats
file_path = Path("data.txt")
if file_path.exists():
    stats = file_path.stat()
    print(f"Size: {stats.st_size} bytes")
    print(f"Modified: {stats.st_mtime}")

# Rename/move file
old_path = Path("old_name.txt")
new_path = Path("new_name.txt")
if old_path.exists():
    old_path.rename(new_path)

# Delete file
file_path = Path("temp.txt")
if file_path.exists():
    file_path.unlink()  # Delete file

Practical File System Examples

from pathlib import Path
import os

# Example 1: Find all Python files in project
def find_python_files(directory):
    """Find all .py files recursively"""
    path = Path(directory)
    return list(path.rglob("*.py"))

py_files = find_python_files(".")
print(f"Found {len(py_files)} Python files")
for file in py_files[:5]:  # Show first 5
    print(f"  {file}")

# Example 2: Get total size of directory
def get_directory_size(directory):
    """Calculate total size of all files in directory"""
    path = Path(directory)
    total = 0
    for file in path.rglob("*"):
        if file.is_file():
            total += file.stat().st_size
    return total

size = get_directory_size(".")
print(f"Directory size: {size / 1024 / 1024:.2f} MB")

# Example 3: Organize files by extension
def organize_by_extension(source_dir):
    """Move files into subdirectories by extension"""
    source = Path(source_dir)

    for file in source.iterdir():
        if file.is_file():
            # Get extension (without dot)
            ext = file.suffix[1:] if file.suffix else "no_extension"

            # Create directory for this extension
            ext_dir = source / ext
            ext_dir.mkdir(exist_ok=True)

            # Move file
            new_path = ext_dir / file.name
            file.rename(new_path)
            print(f"Moved {file.name} to {ext}/ directory")

# organize_by_extension("downloads")

# Example 4: Backup files with timestamp
from datetime import datetime

def backup_file(file_path):
    """Create timestamped backup of file"""
    path = Path(file_path)
    if not path.exists():
        return

    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    backup_name = f"{path.stem}_{timestamp}{path.suffix}"
    backup_path = path.parent / "backups" / backup_name

    # Create backups directory
    backup_path.parent.mkdir(exist_ok=True)

    # Copy file content
    backup_path.write_bytes(path.read_bytes())
    print(f"Backup created: {backup_path}")

# backup_file("important.txt")

# Example 5: Find largest files
def find_largest_files(directory, n=5):
    """Find the N largest files in directory"""
    path = Path(directory)
    files_with_size = []

    for file in path.rglob("*"):
        if file.is_file():
            size = file.stat().st_size
            files_with_size.append((file, size))

    # Sort by size (descending)
    files_with_size.sort(key=lambda x: x[1], reverse=True)

    return files_with_size[:n]

largest = find_largest_files(".", n=5)
print("\nLargest files:")
for file, size in largest:
    print(f"  {size / 1024:.1f} KB - {file}")

Best practice: Use pathlibfor new code, it's more modern, readable, and cross-platform. Use os when you need specific system-level operations or when working with legacy code.

Quick Reference: When to Use Each Module

datetime

Use when you need to:

  • Track timestamps
  • Schedule events
  • Calculate time differences
  • Format dates for display
  • Parse date strings
  • Log with timestamps
json

Use when you need to:

  • Work with APIs
  • Save configuration files
  • Store structured data
  • Exchange data between systems
  • Serialize Python objects
  • Parse JSON responses
collections

Use when you need to:

  • Counter: Count frequencies
  • Group data efficiently
  • deque: Implement queues/stacks (Intermediate)
  • defaultdict: Auto-initialize values (Intermediate)
os & pathlib

Use when you need to:

  • Navigate file systems
  • Create/delete directories
  • Check if files exist
  • Find files by pattern
  • Get file metadata
  • Build cross-platform paths

Key Takeaways

datetime Module

  • datetime.now() for current time
  • strftime() to format dates
  • strptime() to parse dates
  • timedelta for date arithmetic
  • Use for timestamps, scheduling, logging

json Module

  • json.dumps() Python → JSON string
  • json.loads() JSON string → Python
  • json.dump() write to file
  • json.load() read from file
  • Use for APIs, configs, data exchange

collections Module

  • Counter for frequency counting
  • defaultdict for default values (Intermediate)
  • deque for fast both-end ops (Intermediate)
  • most_common() for top items
  • Better than plain dict/list for special cases

os & pathlib

  • Path for modern path handling
  • / operator to join paths
  • .glob() to find files by pattern
  • .exists(), .is_file(), .is_dir()
  • Prefer pathlib for new code
What's Next?

Congratulations! You've learned essential standard library modules - datetime, json, collections, os, and pathlib - that you'll use in virtually every Python project. In the next lesson, we'll explore:

  • Debugging basics - Learn print debugging, using Python's debugger (pdb), and techniques to find and fix bugs efficiently
  • Common error patterns - Understand typical mistakes and how to avoid them
  • Python Intermediate - When ready, explore testing, regular expressions, advanced file formats, and professional development patterns