Standard Library Essentials
Master the powerful tools that come built-in with Python
Introduction
Python's standard library is often called "batteries included", it comes with powerful modules for common tasks that you'll use in almost every project. From handling dates and times to working with JSON data, managing files, and using specialized data structures, these built-in tools save you from reinventing the wheel. Learning the standard library makes you immediately more productive and prepares you for real-world Python development. No installation required, these tools are ready to use right now!
The datetime Module
The datetime module provides classes for working with dates and times. It's essential for scheduling, logging, data analysis, and any application that needs to track when things happen.
Getting Current Date and Time
from datetime import datetime, date, time
# Get current date and time
now = datetime.now()
print(now) # 2024-01-15 14:30:45.123456
# Get just the date
today = date.today()
print(today) # 2024-01-15
# Access individual components
print(now.year) # 2024
print(now.month) # 1
print(now.day) # 15
print(now.hour) # 14
print(now.minute) # 30
print(now.second) # 45
# Get day of week (0 = Monday, 6 = Sunday)
print(today.weekday()) # 0 (if it's Monday)
# Get day name
print(today.strftime("%A")) # "Monday"Creating Specific Dates and Times
from datetime import datetime, date, time # Create specific date birthday = date(1990, 5, 15) # May 15, 1990 print(birthday) # 1990-05-15 # Create specific datetime meeting = datetime(2024, 12, 25, 14, 30) # Dec 25, 2024 at 2:30 PM print(meeting) # 2024-12-25 14:30:00 # Create specific time alarm = time(7, 30, 0) # 7:30:00 AM print(alarm) # 07:30:00 # Create time with microseconds precise_time = time(12, 30, 45, 123456) print(precise_time) # 12:30:45.123456
Formatting Dates and Times
Use strftime() (string format time) to convert dates to readable strings.
from datetime import datetime
now = datetime.now()
# Common formats
print(now.strftime("%Y-%m-%d")) # "2024-01-15"
print(now.strftime("%m/%d/%Y")) # "01/15/2024"
print(now.strftime("%B %d, %Y")) # "January 15, 2024"
print(now.strftime("%A, %B %d, %Y")) # "Monday, January 15, 2024"
# Time formats
print(now.strftime("%H:%M:%S")) # "14:30:45" (24-hour)
print(now.strftime("%I:%M %p")) # "02:30 PM" (12-hour)
# Combined
print(now.strftime("%Y-%m-%d %H:%M:%S")) # "2024-01-15 14:30:45"
print(now.strftime("%B %d, %Y at %I:%M %p"))
# "January 15, 2024 at 02:30 PM"
# Common format codes:
# %Y - Year (4 digits) %m - Month (01-12)
# %d - Day (01-31) %H - Hour 24h (00-23)
# %I - Hour 12h (01-12) %M - Minute (00-59)
# %S - Second (00-59) %p - AM/PM
# %A - Weekday full %B - Month full
# %a - Weekday short %b - Month shortParsing Date Strings
Use strptime() (string parse time) to convert strings to datetime objects.
from datetime import datetime
# Parse different date formats
date_str1 = "2024-01-15"
dt1 = datetime.strptime(date_str1, "%Y-%m-%d")
print(dt1) # 2024-01-15 00:00:00
date_str2 = "01/15/2024"
dt2 = datetime.strptime(date_str2, "%m/%d/%Y")
print(dt2) # 2024-01-15 00:00:00
date_str3 = "January 15, 2024"
dt3 = datetime.strptime(date_str3, "%B %d, %Y")
print(dt3) # 2024-01-15 00:00:00
# Parse datetime with time
datetime_str = "2024-01-15 14:30:45"
dt4 = datetime.strptime(datetime_str, "%Y-%m-%d %H:%M:%S")
print(dt4) # 2024-01-15 14:30:45
# Handle different formats
formats = ["%Y-%m-%d", "%m/%d/%Y", "%d-%m-%Y"]
date_string = "15-01-2024"
for fmt in formats:
try:
parsed = datetime.strptime(date_string, fmt)
print(f"Success: {parsed}")
break
except ValueError:
continueDate Arithmetic
Use timedelta to perform arithmetic with dates.
from datetime import datetime, timedelta, date
# Create timedelta
one_day = timedelta(days=1)
one_week = timedelta(weeks=1)
one_hour = timedelta(hours=1)
mixed = timedelta(days=7, hours=2, minutes=30)
# Add/subtract time
today = date.today()
tomorrow = today + timedelta(days=1)
yesterday = today - timedelta(days=1)
next_week = today + timedelta(weeks=1)
print(f"Today: {today}")
print(f"Tomorrow: {tomorrow}")
print(f"Yesterday: {yesterday}")
print(f"Next week: {next_week}")
# Calculate age
birthday = date(1990, 5, 15)
today = date.today()
age_delta = today - birthday
age_years = age_delta.days // 365
print(f"Age: approximately {age_years} years")
# Calculate days until event
event_date = date(2024, 12, 25)
today = date.today()
days_until = (event_date - today).days
print(f"Days until Christmas: {days_until}")Practical datetime Examples
from datetime import datetime, timedelta
# Calculate project deadline
start_date = datetime(2024, 1, 15)
duration_days = 30
deadline = start_date + timedelta(days=duration_days)
print(f"Project deadline: {deadline.strftime('%B %d, %Y')}")
# "Project deadline: February 14, 2024"
# Check if date is weekend
def is_weekend(date):
return date.weekday() >= 5 # 5 = Saturday, 6 = Sunday
today = datetime.now().date()
if is_weekend(today):
print("It's the weekend!")
else:
print("It's a weekday")
# Calculate business days between dates
def business_days_between(start, end):
days = 0
current = start
while current <= end:
if current.weekday() < 5: # Monday-Friday
days += 1
current += timedelta(days=1)
return days
start = date(2024, 1, 15) # Monday
end = date(2024, 1, 19) # Friday
print(f"Business days: {business_days_between(start, end)}")
# Business days: 5
# Format timestamps for logs
def log_message(message):
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
return f"[{timestamp}] {message}"
print(log_message("Application started"))
# "[2024-01-15 14:30:45] Application started"
# Calculate time remaining
deadline = datetime(2024, 12, 31, 23, 59, 59)
now = datetime.now()
remaining = deadline - now
print(f"Time remaining: {remaining.days} days, {remaining.seconds // 3600} hours")Timezone Note: The examples above use naive datetime objects (no timezone info). For production apps with multiple timezones, use the pytz or zoneinfomodules for timezone-aware datetimes.
The json Module
JSON (JavaScript Object Notation) is the most common data format for APIs and configuration files. The json module makes it easy to convert between Python objects and JSON strings.
Converting Python to JSON (Serialization)
Use json.dumps() (dump string) to convert Python objects to JSON strings.
import json
# Python dictionary to JSON
person = {
"name": "Alice",
"age": 30,
"city": "NYC",
"is_student": False,
"grades": [85, 90, 92]
}
json_string = json.dumps(person)
print(json_string)
# {"name": "Alice", "age": 30, "city": "NYC", "is_student": false, "grades": [85, 90, 92]}
# Pretty print with indentation
json_pretty = json.dumps(person, indent=4)
print(json_pretty)
# {
# "name": "Alice",
# "age": 30,
# "city": "NYC",
# "is_student": false,
# "grades": [85, 90, 92]
# }
# Convert list to JSON
numbers = [1, 2, 3, 4, 5]
json_list = json.dumps(numbers)
print(json_list) # [1, 2, 3, 4, 5]
# Nested structures
data = {
"users": [
{"id": 1, "name": "Alice"},
{"id": 2, "name": "Bob"}
],
"total": 2
}
json_nested = json.dumps(data, indent=2)
print(json_nested)Converting JSON to Python (Deserialization)
Use json.loads() (load string) to convert JSON strings to Python objects.
import json
# JSON string to Python
json_string = '{"name": "Alice", "age": 30, "city": "NYC"}'
person = json.loads(json_string)
print(person) # {"name": "Alice", "age": 30, "city": "NYC"}
print(type(person)) # <class 'dict'>
# Access the data
print(person["name"]) # "Alice"
print(person["age"]) # 30
# JSON array to Python list
json_array = '[1, 2, 3, 4, 5]'
numbers = json.loads(json_array)
print(numbers) # [1, 2, 3, 4, 5]
print(type(numbers)) # <class 'list'>
# Complex nested structure
json_data = '''
{
"users": [
{"id": 1, "name": "Alice", "active": true},
{"id": 2, "name": "Bob", "active": false}
],
"total": 2,
"page": 1
}
'''
data = json.loads(json_data)
print(data["users"][0]["name"]) # "Alice"
print(len(data["users"])) # 2Reading and Writing JSON Files
import json
# Write Python object to JSON file
data = {
"name": "Alice",
"age": 30,
"hobbies": ["reading", "coding", "hiking"]
}
with open("person.json", "w") as file:
json.dump(data, file, indent=4) # dump (no 's') for files
# Creates person.json with formatted content
# Read JSON file to Python object
with open("person.json", "r") as file:
loaded_data = json.load(file) # load (no 's') for files
print(loaded_data["name"]) # "Alice"
print(loaded_data["hobbies"]) # ["reading", "coding", "hiking"]
# Append to existing JSON file (read, modify, write)
with open("person.json", "r") as file:
data = json.load(file)
data["city"] = "NYC" # Add new field
data["age"] = 31 # Update field
with open("person.json", "w") as file:
json.dump(data, file, indent=4)Practical JSON Examples
import json
# Configuration file
config = {
"database": {
"host": "localhost",
"port": 5432,
"name": "myapp"
},
"debug": True,
"max_connections": 100
}
# Save configuration
with open("config.json", "w") as f:
json.dump(config, f, indent=4)
# Load configuration
with open("config.json", "r") as f:
loaded_config = json.load(f)
db_host = loaded_config["database"]["host"]
print(f"Database host: {db_host}")
# API response simulation
api_response = {
"status": "success",
"data": {
"users": [
{"id": 1, "username": "alice", "email": "alice@example.com"},
{"id": 2, "username": "bob", "email": "bob@example.com"}
]
},
"timestamp": "2024-01-15T14:30:00Z"
}
# Convert to JSON string (as API would return)
json_response = json.dumps(api_response)
# Parse JSON response (as client would receive)
parsed = json.loads(json_response)
for user in parsed["data"]["users"]:
print(f"{user['username']}: {user['email']}")
# alice: alice@example.com
# bob: bob@example.com
# Handle JSON errors
invalid_json = '{"name": "Alice", "age": }' # Missing value
try:
data = json.loads(invalid_json)
except json.JSONDecodeError as e:
print(f"Invalid JSON: {e}")
# Invalid JSON: Expecting value: line 1 column 25 (char 24)JSON Type Mapping: Python's dict ↔ JSON object, list/tuple ↔ JSON array, str ↔ JSON string, int/float ↔ JSON number, True/False ↔ JSON true/false, None ↔ JSON null
The collections Module
The collections module provides specialized container data types that extend Python's built-in containers (dict, list, set, tuple) with additional functionality.
Counter - Count Occurrences
Counter is a dict subclass for counting hashable objects. Perfect for frequency analysis.
from collections import Counter
# Count items in a list
fruits = ["apple", "banana", "apple", "cherry", "banana", "apple"]
counter = Counter(fruits)
print(counter)
# Counter({"apple": 3, "banana": 2, "cherry": 1})
# Access counts
print(counter["apple"]) # 3
print(counter["cherry"]) # 1
print(counter["grape"]) # 0 (no KeyError!)
# Count characters in a string
text = "hello world"
char_count = Counter(text)
print(char_count)
# Counter({"l": 3, "o": 2, "h": 1, "e": 1, " ": 1, "w": 1, "r": 1, "d": 1})
# Most common elements
print(counter.most_common(2)) # [("apple", 3), ("banana", 2)]
# Count words in text
sentence = "the quick brown fox jumps over the lazy dog"
words = sentence.split()
word_count = Counter(words)
print(word_count.most_common(3))
# [("the", 2), ("quick", 1), ("brown", 1)]Counter Operations
from collections import Counter
# Create Counter from different sources
c1 = Counter([1, 2, 2, 3, 3, 3])
c2 = Counter({"a": 3, "b": 2})
c3 = Counter(a=3, b=2) # keyword arguments
# Arithmetic operations
c1 = Counter(a=3, b=1)
c2 = Counter(a=1, b=2)
# Addition
combined = c1 + c2
print(combined) # Counter({"a": 4, "b": 3})
# Subtraction (keeps positive counts only)
diff = c1 - c2
print(diff) # Counter({"a": 2})
# Intersection (minimum)
intersection = c1 & c2
print(intersection) # Counter({"a": 1, "b": 1})
# Union (maximum)
union = c1 | c2
print(union) # Counter({"a": 3, "b": 2})
# Update (add counts)
c1 = Counter(a=3, b=1)
c1.update({"a": 1, "b": 2, "c": 3})
print(c1) # Counter({"a": 4, "b": 3, "c": 3})
# Get total count
total = sum(c1.values())
print(f"Total items: {total}") # 10Working with Files and Paths: os and pathlib
The os module provides operating system functionality, while pathlib offers an object-oriented approach to file paths. Both are essential for file and directory operations.
Basic os Module Operations
import os
# Get current working directory
cwd = os.getcwd()
print(f"Current directory: {cwd}")
# /home/user/projects
# Change directory
os.chdir("/tmp")
print(f"New directory: {os.getcwd()}")
# /tmp
# List files in directory
files = os.listdir(".")
print(files)
# ["file1.txt", "file2.py", "folder1"]
# Check if path exists
print(os.path.exists("file1.txt")) # True
print(os.path.exists("nonexistent")) # False
# Check if it's a file or directory
print(os.path.isfile("file1.txt")) # True
print(os.path.isdir("folder1")) # True
# Get file size (in bytes)
size = os.path.getsize("file1.txt")
print(f"Size: {size} bytes")
# Create directory
os.mkdir("new_folder") # Single directory
os.makedirs("parent/child/grandchild") # Nested directories
# Remove directory
os.rmdir("new_folder") # Empty directory only
# Use shutil.rmtree() for non-empty directoriesWorking with Paths (os.path)
import os
# Join paths (handles OS differences)
path = os.path.join("folder", "subfolder", "file.txt")
print(path)
# Linux/Mac: folder/subfolder/file.txt
# Windows: folder\subfolder\file.txt
# Split path into directory and filename
full_path = "/home/user/documents/report.pdf"
directory, filename = os.path.split(full_path)
print(f"Directory: {directory}") # /home/user/documents
print(f"Filename: {filename}") # report.pdf
# Get filename without path
basename = os.path.basename(full_path)
print(basename) # report.pdf
# Get directory without filename
dirname = os.path.dirname(full_path)
print(dirname) # /home/user/documents
# Split filename and extension
name, ext = os.path.splitext("report.pdf")
print(f"Name: {name}") # report
print(f"Extension: {ext}") # .pdf
# Get absolute path
rel_path = "data/file.txt"
abs_path = os.path.abspath(rel_path)
print(abs_path) # /home/user/projects/data/file.txt
# Check if path is absolute
print(os.path.isabs("/home/user")) # True
print(os.path.isabs("data/file.txt")) # FalseModern Path Handling with pathlib
pathlib provides an object-oriented interface that's more intuitive and Pythonic than os.path.
from pathlib import Path
# Create Path object
path = Path("folder/subfolder/file.txt")
print(path) # folder/subfolder/file.txt
# Current directory
cwd = Path.cwd()
print(cwd) # /home/user/projects
# Home directory
home = Path.home()
print(home) # /home/user
# Join paths with / operator
data_dir = Path("data")
file_path = data_dir / "reports" / "2024" / "january.csv"
print(file_path) # data/reports/2024/january.csv
# Check existence
print(file_path.exists()) # False (if doesn't exist)
print(file_path.is_file()) # False
print(file_path.is_dir()) # False
# Get parts of path
full_path = Path("/home/user/documents/report.pdf")
print(full_path.parent) # /home/user/documents
print(full_path.name) # report.pdf
print(full_path.stem) # report
print(full_path.suffix) # .pdf
# Get all parent directories
print(full_path.parents[0]) # /home/user/documents
print(full_path.parents[1]) # /home/user
print(full_path.parents[2]) # /homeFile Operations with pathlib
from pathlib import Path
# Read file
file_path = Path("data.txt")
if file_path.exists():
content = file_path.read_text()
print(content)
# Write file
file_path.write_text("Hello, World!")
# Read/write bytes
binary_data = file_path.read_bytes()
file_path.write_bytes(b"Binary content")
# Create directory
new_dir = Path("new_folder")
new_dir.mkdir(exist_ok=True) # No error if exists
# Create nested directories
nested = Path("parent/child/grandchild")
nested.mkdir(parents=True, exist_ok=True)
# List all files in directory
data_dir = Path("data")
for file in data_dir.iterdir():
print(file)
# Find files by pattern (glob)
# Find all .txt files
txt_files = list(data_dir.glob("*.txt"))
print(txt_files)
# Find all .py files recursively
py_files = list(data_dir.rglob("*.py"))
print(py_files)
# Get file stats
file_path = Path("data.txt")
if file_path.exists():
stats = file_path.stat()
print(f"Size: {stats.st_size} bytes")
print(f"Modified: {stats.st_mtime}")
# Rename/move file
old_path = Path("old_name.txt")
new_path = Path("new_name.txt")
if old_path.exists():
old_path.rename(new_path)
# Delete file
file_path = Path("temp.txt")
if file_path.exists():
file_path.unlink() # Delete filePractical File System Examples
from pathlib import Path
import os
# Example 1: Find all Python files in project
def find_python_files(directory):
"""Find all .py files recursively"""
path = Path(directory)
return list(path.rglob("*.py"))
py_files = find_python_files(".")
print(f"Found {len(py_files)} Python files")
for file in py_files[:5]: # Show first 5
print(f" {file}")
# Example 2: Get total size of directory
def get_directory_size(directory):
"""Calculate total size of all files in directory"""
path = Path(directory)
total = 0
for file in path.rglob("*"):
if file.is_file():
total += file.stat().st_size
return total
size = get_directory_size(".")
print(f"Directory size: {size / 1024 / 1024:.2f} MB")
# Example 3: Organize files by extension
def organize_by_extension(source_dir):
"""Move files into subdirectories by extension"""
source = Path(source_dir)
for file in source.iterdir():
if file.is_file():
# Get extension (without dot)
ext = file.suffix[1:] if file.suffix else "no_extension"
# Create directory for this extension
ext_dir = source / ext
ext_dir.mkdir(exist_ok=True)
# Move file
new_path = ext_dir / file.name
file.rename(new_path)
print(f"Moved {file.name} to {ext}/ directory")
# organize_by_extension("downloads")
# Example 4: Backup files with timestamp
from datetime import datetime
def backup_file(file_path):
"""Create timestamped backup of file"""
path = Path(file_path)
if not path.exists():
return
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
backup_name = f"{path.stem}_{timestamp}{path.suffix}"
backup_path = path.parent / "backups" / backup_name
# Create backups directory
backup_path.parent.mkdir(exist_ok=True)
# Copy file content
backup_path.write_bytes(path.read_bytes())
print(f"Backup created: {backup_path}")
# backup_file("important.txt")
# Example 5: Find largest files
def find_largest_files(directory, n=5):
"""Find the N largest files in directory"""
path = Path(directory)
files_with_size = []
for file in path.rglob("*"):
if file.is_file():
size = file.stat().st_size
files_with_size.append((file, size))
# Sort by size (descending)
files_with_size.sort(key=lambda x: x[1], reverse=True)
return files_with_size[:n]
largest = find_largest_files(".", n=5)
print("\nLargest files:")
for file, size in largest:
print(f" {size / 1024:.1f} KB - {file}")Best practice: Use pathlibfor new code, it's more modern, readable, and cross-platform. Use os when you need specific system-level operations or when working with legacy code.
Quick Reference: When to Use Each Module
datetime
Use when you need to:
- Track timestamps
- Schedule events
- Calculate time differences
- Format dates for display
- Parse date strings
- Log with timestamps
json
Use when you need to:
- Work with APIs
- Save configuration files
- Store structured data
- Exchange data between systems
- Serialize Python objects
- Parse JSON responses
collections
Use when you need to:
- Counter: Count frequencies
- Group data efficiently
- deque: Implement queues/stacks (Intermediate)
- defaultdict: Auto-initialize values (Intermediate)
os & pathlib
Use when you need to:
- Navigate file systems
- Create/delete directories
- Check if files exist
- Find files by pattern
- Get file metadata
- Build cross-platform paths
Key Takeaways
datetime Module
datetime.now()for current timestrftime()to format datesstrptime()to parse datestimedeltafor date arithmetic- Use for timestamps, scheduling, logging
json Module
json.dumps()Python → JSON stringjson.loads()JSON string → Pythonjson.dump()write to filejson.load()read from file- Use for APIs, configs, data exchange
collections Module
Counterfor frequency countingdefaultdictfor default values (Intermediate)dequefor fast both-end ops (Intermediate)most_common()for top items- Better than plain dict/list for special cases
os & pathlib
Pathfor modern path handling/operator to join paths.glob()to find files by pattern.exists(),.is_file(),.is_dir()- Prefer pathlib for new code
What's Next?
Congratulations! You've learned essential standard library modules - datetime, json, collections, os, and pathlib - that you'll use in virtually every Python project. In the next lesson, we'll explore:
- Debugging basics - Learn print debugging, using Python's debugger (pdb), and techniques to find and fix bugs efficiently
- Common error patterns - Understand typical mistakes and how to avoid them
- Python Intermediate - When ready, explore testing, regular expressions, advanced file formats, and professional development patterns