Security Testing & Auditing
SAST, DAST, bandit, pip audit, Fuzzing, Pen Testing
Introduction
You can't fix what you can't find. Security testing proactively discovers vulnerabilities before attackers do. This lesson covers the automated tools Python developers should run in every CI/CD pipeline, the property-based testing approach for security edge cases, and the structured methodology for penetration testing. You'll build an automated security audit script that runs all checks in one pass.
SAST vs DAST
SAST, Static Application Security Testing
Analyzes source code without executing it. Finds issues early in development.
- Runs on source code, bytecode, or binary
- Fast, runs in CI/CD on every commit
- No running application needed
- High false positive rate
DAST, Dynamic Application Security Testing
Tests running applications by sending malicious inputs. Finds runtime issues.
- Requires deployed application
- Finds runtime/configuration issues
- Simulates real attacker behavior
- Slower, run on staging, not every commit
bandit, Python SAST Tool
bandit is an open-source Static Application Security Testing (SAST) tool built specifically for Python. It parses each source file into an Abstract Syntax Tree (AST) and runs a set of plugins (called test IDs, e.g. B602, B324) against the tree to detect patterns that are known to be dangerous: shell injection, use of weak hash functions, hard-coded passwords, insecure use of pickle, yaml.load, and many more. bandit never runs the code, it only reads it, so it is safe to run on untrusted codebases and fast enough to include in pre-commit hooks and CI/CD pipelines.
Severity Levels
- HIGH, Exploit is straightforward and impact is severe (e.g.
shell=Truewith user input) - MEDIUM, Issue is real but exploitability depends on context (e.g. MD5 used for passwords)
- LOW, Minor concern or best-practice deviation (e.g.
assertin non-test code)
Common Test IDs
B101- use ofassert(stripped in optimized bytecode)B311-randommodule used for security purposesB324- weak hash function (MD5, SHA1)B506-yaml.load()withoutLoader(arbitrary code exec)B602/B603- subprocess with shell injection risk
Install: pip install bandit
# Run on your project
bandit -r ./src -f json -o security_report.json
# Run with specific severity threshold
bandit -r ./src -l -ll # Only show MEDIUM and HIGH severity
# ── Example vulnerable code that bandit catches ───────────────
# vulnerable_code.py
import subprocess
import hashlib
import random
# B603: subprocess without shell=True but with user data, flagged
user_input = "untrusted data"
subprocess.run(user_input, shell=True) # B602: HIGH severity
# B324: hashlib with MD5, flagged
hash_value = hashlib.md5(b"password").hexdigest() # B324: MEDIUM severity
# B311: Standard pseudo-random generators, flagged for security use
token = str(random.randint(0, 1000000)) # B311: MEDIUM severity
# Fix: use secrets.token_hex() instead
# ── Interpret bandit JSON output ──────────────────────────────
import json
example_output = {
"results": [
{
"filename": "vulnerable_code.py",
"line_number": 8,
"test_id": "B602",
"issue_text": "subprocess call with shell=True identified, security issue.",
"issue_severity": "HIGH",
"issue_confidence": "HIGH",
},
{
"filename": "vulnerable_code.py",
"line_number": 11,
"test_id": "B324",
"issue_text": "Use of insecure MD2, MD4, MD5, or SHA1 hash function.",
"issue_severity": "MEDIUM",
"issue_confidence": "HIGH",
},
]
}
for issue in example_output["results"]:
severity = issue["issue_severity"]
color = "❌" if severity == "HIGH" else "⚠️"
print(f"{color} [{severity}] {issue['test_id']} line {issue['line_number']}: {issue['issue_text'][:60]}")
Expected Output:
❌ [HIGH] B602 line 8: subprocess call with shell=True identified, security issue. ⚠️ [MEDIUM] B324 line 11: Use of insecure MD2, MD4, MD5, or SHA1 hash function.
Fuzzing with Hypothesis
Fuzzing feeds random/unexpected inputs to your code to find crashes, exceptions, and security edge cases. Python's Hypothesis library does property-based fuzzing.
Install: pip install hypothesis pytest
# fuzz_test.py
from hypothesis import given, settings, HealthCheck
from hypothesis import strategies as st
import re
# Function to test
def parse_user_input(data: str) -> dict:
"""Parse a user-supplied key=value string."""
if not isinstance(data, str):
raise TypeError("Input must be string")
if len(data) > 1000:
raise ValueError("Input too long")
result = {}
for part in data.split("&"):
if "=" in part:
key, _, value = part.partition("=")
key = key.strip()
if re.match(r'^[a-zA-Z0-9_]+$', key):
result[key] = value
return result
# Property-based security tests
@given(st.text())
@settings(max_examples=1000, suppress_health_check=[HealthCheck.too_slow])
def test_no_crash_on_arbitrary_input(data: str):
"""Function must not crash on ANY string input, only raise ValueError/TypeError."""
try:
result = parse_user_input(data)
# If it returns, result must be a dict
assert isinstance(result, dict)
except (ValueError, TypeError):
pass # Expected errors are OK
# Any other exception (KeyError, IndexError, etc.) = bug!
@given(st.text(alphabet="<>"';&", min_size=1, max_size=100))
def test_injection_chars_handled_safely(injection_payload: str):
"""Injection characters must not cause unexpected behavior."""
try:
result = parse_user_input(injection_payload)
# Output dict keys must only contain safe characters
for key in result:
assert re.match(r'^[a-zA-Z0-9_]+$', key), f"Unsafe key in output: {key!r}"
except (ValueError, TypeError):
pass
# Run: pytest fuzz_test.py -v
print("Hypothesis will generate 1000+ test cases automatically")
print("It specifically tries edge cases: empty strings, unicode, very long strings, etc.")
Automated Security Audit Script
# security_audit.py, Run all security checks in one script
import subprocess
import sys
import json
from pathlib import Path
def run_bandit(path: str) -> tuple[bool, int]:
"""Run bandit SAST. Returns (passed, high_severity_count)."""
try:
result = subprocess.run(
["bandit", "-r", path, "-f", "json", "-q"],
capture_output=True, text=True, timeout=60
)
data = json.loads(result.stdout or "{}")
high_issues = sum(1 for i in data.get("results", [])
if i.get("issue_severity") == "HIGH")
return high_issues == 0, high_issues
except (FileNotFoundError, json.JSONDecodeError):
return False, -1
def run_pip_audit() -> tuple[bool, int]:
"""Run pip-audit for CVE scanning."""
try:
result = subprocess.run(
["pip", "audit", "--format=json"],
capture_output=True, text=True, timeout=120
)
data = json.loads(result.stdout or "[]")
vuln_count = len(data)
return vuln_count == 0, vuln_count
except (FileNotFoundError, json.JSONDecodeError):
return False, -1
def check_env_file_not_in_git() -> bool:
"""Ensure .env file is in .gitignore."""
gitignore = Path(".gitignore")
if not gitignore.exists():
return False
content = gitignore.read_text()
return ".env" in content
def run_full_audit(source_path: str = "./src") -> bool:
print("=" * 50)
print(" SECURITY AUDIT REPORT")
print("=" * 50)
checks = []
passed, highs = run_bandit(source_path)
checks.append(("bandit SAST (no HIGH issues)", passed,
f"{highs} HIGH issues" if not passed else "Clean"))
passed, vulns = run_pip_audit()
checks.append(("pip-audit (no CVEs)", passed,
f"{vulns} vulnerabilities" if not passed else "Clean"))
env_safe = check_env_file_not_in_git()
checks.append((".env in .gitignore", env_safe, "Add .env to .gitignore" if not env_safe else ""))
total_passed = sum(1 for _, p, _ in checks if p)
for name, passed, detail in checks:
icon = "✅" if passed else "❌"
print(f" {icon} {name}")
if not passed and detail:
print(f" → {detail}")
print(f"\nResult: {total_passed}/{len(checks)} checks passed")
return total_passed == len(checks)
success = run_full_audit()
sys.exit(0 if success else 1)
Expected Output:
================================================== SECURITY AUDIT REPORT ================================================== ✅ bandit SAST (no HIGH issues) ✅ pip-audit (no CVEs) ✅ .env in .gitignore Result: 3/3 checks passed
Key Takeaways
- Run bandit in CI/CD, block merges that introduce HIGH severity issues; make it automatic, not optional
- Run pip audit on every build, CVEs in dependencies are the most common source of production security incidents
- SAST finds code issues, DAST finds runtime issues, use both; they catch different vulnerability classes
- Hypothesis tests edge cases automatically, property-based testing finds inputs that humans would never think to try
- Shift left, security testing in development is 10× cheaper than finding issues in production
- Security audit scripts belong in CI/CD, automated checks are more reliable than manual review checklists
What's Next?
Lesson 17 moves to infrastructure security, securing Docker containers and Kubernetes clusters against both external attackers and insider threats.
- Docker security baseline, non-root, read-only FS, capability dropping
- Image scanning with trivy, CVE scanning for container images
- Kubernetes RBAC and NetworkPolicy, cluster-level access control
- AWS IAM least privilege, boto3-based permission checks