Advanced Queries

Subqueries, CTEs, and window functions

Beyond Basic Queries

Once you master basic SELECT statements, joins, and aggregations, you're ready for advanced querying techniques. Subqueries let you nest queries inside queries. CTEs (Common Table Expressions) make complex queries readable and reusable. Window functions perform calculations across rows without grouping them. These tools unlock powerful analytical capabilities and help you write cleaner, more maintainable SQL.

Subqueries

A subquery is a query nested inside another query. Use it to break complex problems into smaller, logical steps.

Subquery in WHERE Clause

Find employees who earn more than the average salary.

SELECT name, salary
FROM employees
WHERE salary > (SELECT AVG(salary) FROM employees);
Result:
Alice | $85,000
Bob | $92,000
(Average was $70,000)

Subquery in SELECT Clause

Show each employee with the department's average salary.

SELECT
    name,
    salary,
    (
        SELECT AVG(salary)
        FROM employees e2
        WHERE e2.dept_id = e1.dept_id
    ) AS dept_avg
FROM employees e1;
Result:
Alice | $85,000 | $78,500 (Engineering avg)
Bob | $72,000 | $78,500 (Engineering avg)
Carol | $65,000 | $67,000 (Sales avg)

Subquery in FROM Clause

Find departments with average salary above $75,000.

SELECT dept_name, avg_salary
FROM (
    SELECT dept_id, AVG(salary) AS avg_salary
    FROM employees
    GROUP BY dept_id
) AS dept_averages
JOIN departments ON departments.id = dept_averages.dept_id
WHERE avg_salary > 75000;
Result:
Engineering | $78,500
Product | $82,000

Subquery with IN

Find customers who have placed orders in the last 30 days.

SELECT name, email
FROM customers
WHERE customer_id IN (
    SELECT DISTINCT customer_id
    FROM orders
    WHERE order_date >= CURRENT_DATE - INTERVAL '30 days'
);
Result:
Alice Johnson | alice@example.com
Bob Smith | bob@example.com
(Only customers with recent orders)

Subquery with EXISTS

Find products that have never been ordered.

SELECT product_name, price
FROM products p
WHERE NOT EXISTS (
    SELECT 1
    FROM order_items oi
    WHERE oi.product_id = p.product_id
);
Result:
Vintage Camera | $899
Standing Desk | $549
(Products with 0 orders)
💡 Tip: Use EXISTS instead of IN when checking for existence, it's often faster because it stops searching once it finds a match.

Common Table Expressions (CTEs)

CTEs (WITH clauses) create temporary named result sets that exist only for the duration of a query. They make complex queries more readable and maintainable.

Basic CTE

Find high-value customers (those who spent over $1,000).

WITH customer_totals AS (
    SELECT
        customer_id,
        SUM(total) AS total_spent
    FROM orders
    GROUP BY customer_id
)
SELECT c.name, ct.total_spent
FROM customer_totals ct
JOIN customers c ON c.customer_id = ct.customer_id
WHERE ct.total_spent > 1000
ORDER BY ct.total_spent DESC;
Result:
Alice Johnson | $2,450
Bob Smith | $1,850
Carol White | $1,320

Multiple CTEs

Compare department averages to company average.

WITH dept_averages AS (
    SELECT
        dept_id,
        AVG(salary) AS avg_salary
    FROM employees
    GROUP BY dept_id
),
company_average AS (
    SELECT AVG(salary) AS company_avg
    FROM employees
)
SELECT
    d.dept_name,
    da.avg_salary,
    ca.company_avg,
    da.avg_salary - ca.company_avg AS difference
FROM dept_averages da
JOIN departments d ON d.dept_id = da.dept_id
CROSS JOIN company_average ca
ORDER BY difference DESC;
Result:
Engineering | $78,500 | $70,000 | +$8,500
Product | $72,000 | $70,000 | +$2,000
Sales | $65,000 | $70,000 | -$5,000

Recursive CTE

Find an employee's entire management chain.

WITH RECURSIVE management_chain AS (
    -- Base case: start with the employee
    SELECT
        employee_id,
        name,
        manager_id,
        0 AS level
    FROM employees
    WHERE employee_id = 42

    UNION ALL

    -- Recursive case: find their manager
    SELECT
        e.employee_id,
        e.name,
        e.manager_id,
        mc.level + 1
    FROM employees e
    JOIN management_chain mc ON e.employee_id = mc.manager_id
)
SELECT level, name
FROM management_chain
ORDER BY level;
Result:
0 | John Doe (starting employee)
1 | Jane Manager
2 | Alice Director
3 | Bob CEO
💡 CTE vs Subquery: Use CTEs when you need to reference the same result set multiple times, or when readability matters. Use subqueries for simple, one-time calculations.

Window Functions

Window functions perform calculations across rows that are related to the current row, without collapsing them into groups. Think of them as "looking through a window" at nearby rows.

ROW_NUMBER()

Assign a unique sequential number to each row.

SELECT
    name,
    salary,
    ROW_NUMBER() OVER (ORDER BY salary DESC) AS rank
FROM employees;
Result:
Bob | $92,000 | 1
Alice | $85,000 | 2
Carol | $72,000 | 3
David | $65,000 | 4

RANK() and DENSE_RANK()

Rank rows with ties. RANK skips numbers after ties, DENSE_RANK doesn't.

SELECT
    name,
    salary,
    RANK() OVER (ORDER BY salary DESC) AS rank,
    DENSE_RANK() OVER (ORDER BY salary DESC) AS dense_rank
FROM employees;
Result:
Bob | $92,000 | 1 | 1
Alice | $85,000 | 2 | 2
Carol | $72,000 | 3 | 3
David | $72,000 | 3 | 3 (tied)
Eve | $65,000 | 5 | 4 (RANK skipped 4)

PARTITION BY

Rank employees within each department separately.

SELECT
    dept_name,
    name,
    salary,
    RANK() OVER (PARTITION BY dept_id ORDER BY salary DESC) AS dept_rank
FROM employees
JOIN departments USING (dept_id);
Result:
Engineering | Alice | $85,000 | 1
Engineering | Bob | $72,000 | 2
Sales | Carol | $65,000 | 1
Sales | David | $62,000 | 2
(Rankings restart for each department)

Running Totals with SUM()

Calculate cumulative sales over time.

SELECT
    order_date,
    total,
    SUM(total) OVER (ORDER BY order_date) AS running_total
FROM orders
ORDER BY order_date;
Result:
2024-01-01 | $100 | $100
2024-01-02 | $150 | $250
2024-01-03 | $200 | $450
2024-01-04 | $175 | $625

Moving Average

Calculate 3-day moving average of sales.

SELECT
    order_date,
    total,
    AVG(total) OVER (
        ORDER BY order_date
        ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
    ) AS moving_avg_3day
FROM orders
ORDER BY order_date;
Result:
2024-01-01 | $100 | $100 (only 1 day)
2024-01-02 | $150 | $125 (2 days)
2024-01-03 | $200 | $150 (3 days: 100+150+200)
2024-01-04 | $175 | $175 (3 days: 150+200+175)

LAG() and LEAD()

Access previous or next row's value.

SELECT
    order_date,
    total,
    LAG(total) OVER (ORDER BY order_date) AS prev_day,
    total - LAG(total) OVER (ORDER BY order_date) AS change
FROM orders
ORDER BY order_date;
Result:
2024-01-01 | $100 | NULL | NULL
2024-01-02 | $150 | $100 | +$50
2024-01-03 | $200 | $150 | +$50
2024-01-04 | $175 | $200 | -$25

NTILE()

Divide rows into N equal groups (quartiles, deciles, etc.).

SELECT
    name,
    salary,
    NTILE(4) OVER (ORDER BY salary DESC) AS quartile
FROM employees;
Result:
Bob | $92,000 | 1 (top 25%)
Alice | $85,000 | 1
Carol | $72,000 | 2
David | $65,000 | 3
Eve | $58,000 | 4 (bottom 25%)

Window Frame Clauses

Frame clauses define which rows the window function considers. They let you control the "window" size.

ROWS

Physical rows relative to current

ROWS BETWEEN 
  2 PRECEDING 
  AND CURRENT ROW
RANGE

Logical range based on values

RANGE BETWEEN 
  UNBOUNDED PRECEDING 
  AND CURRENT ROW

Common Frame Specifications

-- All rows from start to current
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW

-- Current and next row
ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING

-- Previous 3, current, and next 3 rows
ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING

-- All rows in partition
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
Frame clauses control which rows are included in calculations

Real-World Analytics Examples

Top N per Category

Find the top 3 selling products in each category.

WITH ranked_products AS (
    SELECT
        category_name,
        product_name,
        total_sales,
        ROW_NUMBER() OVER (
            PARTITION BY category_id 
            ORDER BY total_sales DESC
        ) AS rank
    FROM products
    JOIN categories USING (category_id)
)
SELECT category_name, product_name, total_sales
FROM ranked_products
WHERE rank <= 3
ORDER BY category_name, rank;
Result:
Electronics | Laptop Pro | $125,000
Electronics | Wireless Mouse | $45,000
Electronics | USB-C Cable | $23,000
Furniture | Office Chair | $89,000
Furniture | Standing Desk | $67,000
Furniture | Monitor Arm | $34,000

Year-over-Year Growth

Compare monthly sales to the same month last year.

SELECT
    month,
    year,
    monthly_sales,
    LAG(monthly_sales, 12) OVER (ORDER BY year, month) AS prev_year_sales,
    ROUND(
      (monthly_sales - LAG(monthly_sales, 12) OVER (ORDER BY year, month))
      / LAG(monthly_sales, 12) OVER (ORDER BY year, month) * 100, 1
    ) AS yoy_growth_pct
FROM monthly_sales_summary
ORDER BY year, month;
Result:
Jan | 2023 | $50,000 | NULL | NULL
Jan | 2024 | $58,000 | $50,000 | +16.0%
Feb | 2024 | $62,000 | $52,000 | +19.2%

Customer Lifetime Value Segments

Segment customers by their total spending.

WITH customer_spending AS (
    SELECT
        customer_id,
        SUM(total) AS lifetime_value,
        NTILE(3) OVER (ORDER BY SUM(total) DESC) AS value_segment
    FROM orders
    GROUP BY customer_id
)
SELECT CASE
    value_segment
        WHEN 1 THEN 'High Value'
        WHEN 2 THEN 'Medium Value'
        WHEN 3 THEN 'Low Value'
    END AS segment,
    COUNT(*) AS customer_count,
    ROUND(AVG(lifetime_value), 2) AS avg_ltv,
    ROUND(MIN(lifetime_value), 2) AS min_ltv,
    ROUND(MAX(lifetime_value), 2) AS max_ltv
FROM customer_spending
GROUP BY value_segment
ORDER BY value_segment;
Result:
High Value | 334 | $2,450.50 | $1,200 | $5,890
Medium Value | 333 | $675.25 | $450 | $1,199
Low Value | 333 | $185.75 | $50 | $449

Cohort Analysis

Analyze customer retention by signup cohort.

WITH first_purchase AS (
    SELECT
        customer_id,
        DATE_TRUNC('month', MIN(order_date)) AS cohort_month
    FROM orders
    GROUP BY customer_id
),
cohort_activity AS (
    SELECT
        fp.cohort_month,
        DATE_TRUNC('month', o.order_date) AS activity_month,
        COUNT(DISTINCT o.customer_id)     AS active_customers
    FROM first_purchase fp
    JOIN orders o ON o.customer_id = fp.customer_id
    GROUP BY fp.cohort_month, DATE_TRUNC('month', o.order_date)
)
SELECT
    cohort_month,
    activity_month,
    active_customers,
    FIRST_VALUE(active_customers) OVER (
        PARTITION BY cohort_month 
        ORDER BY activity_month
    ) AS cohort_size,
    ROUND(
        active_customers::NUMERIC / 
        FIRST_VALUE(active_customers) OVER (
            PARTITION BY cohort_month 
            ORDER BY activity_month
        ) * 100,
      1
    ) AS retention_pct
FROM cohort_activity
ORDER BY cohort_month, activity_month;
Result:
2024-01 | 2024-01 | 100 | 100 | 100.0%
2024-01 | 2024-02 | 65 | 100 | 65.0%
2024-01 | 2024-03 | 52 | 100 | 52.0%
(Shows how Jan 2024 cohort retained over time)

Performance Tips

✅ Index Key Columns

Add indexes on columns used in subquery WHERE clauses and window function ORDER BY clauses for better performance.

✅ Limit Subquery Results

Filter early in subqueries to reduce the amount of data being processed. Don't fetch everything then filter.

✅ Use CTEs for Readability

CTEs make complex queries maintainable and are often optimized as well as equivalent subqueries by modern databases.

✅ Minimize Window Frames

Smaller window frames (e.g., 3 PRECEDING instead of UNBOUNDED PRECEDING) perform better for large datasets.

❌ Avoid Correlated Subqueries

Subqueries that reference outer query columns run once per row and can be very slow. Use JOINs or window functions instead.

❌ Don't Nest Too Deeply

Multiple levels of nested subqueries become hard to read and debug. Break them into CTEs instead.

Common Query Patterns

Pattern 1: Find Duplicates

SELECT email, COUNT(*) AS duplicate_count
FROM users
GROUP BY email
HAVING COUNT(*) > 1;
Result:
john@example.com | 3
jane@example.com | 2

Pattern 2: Running Difference

SELECT
    date,
    inventory_count,
    inventory_count - LAG(inventory_count) OVER (ORDER BY date) AS daily_change
FROM inventory_snapshots
ORDER BY date;
Result:
2024-01-01 | 1000 | NULL
2024-01-02 | 985 | -15
2024-01-03 | 1020 | +35

Pattern 3: Percent of Total

SELECT
    product_name,
    sales,
    ROUND(
            sales * 100.0 / SUM(sales) OVER (),
            2
    ) AS pct_of_total
FROM product_sales
ORDER BY sales DESC;
Result:
Laptop | $50,000 | 35.71%
Mouse | $40,000 | 28.57%
Monitor | $30,000 | 21.43%
Keyboard | $20,000 | 14.29%

Pattern 4: First/Last in Group

WITH ranked_orders AS (
    SELECT
        customer_id,
        order_date,
        total,
        ROW_NUMBER() OVER (
            PARTITION BY customer_id 
            ORDER BY order_date DESC
        ) AS recency_rank
    FROM orders
)
SELECT customer_id, order_date, total
FROM ranked_orders
WHERE recency_rank = 1;
Result (most recent order per customer):
42 | 2024-03-15 | $250
43 | 2024-03-14 | $180
44 | 2024-03-16 | $320

When to Use What

TechniqueUse WhenExample
Subquery in WHEREFiltering based on aggregated dataEmployees above average salary
Subquery in SELECTAdding calculated column from related dataShow department average with each employee
CTEComplex queries, multiple references, readabilityMulti-step analysis with intermediate results
Recursive CTEHierarchical or graph dataOrg charts, bill of materials, category trees
ROW_NUMBER()Unique sequential numberingTop N per group, pagination
RANK()Ranking with gaps for tiesLeaderboards, competition rankings
Running TotalCumulative calculationsYear-to-date sales, cumulative metrics
LAG/LEADCompare to previous/next rowPeriod-over-period growth, changes

Practice Challenge

Challenge: Customer Purchase Analysis

Write a query that shows for each customer:

  • Customer name and total lifetime value
  • Number of orders they've placed
  • Their rank by lifetime value (highest = 1)
  • Date of their first and most recent order
  • Average days between orders
  • Whether they've ordered in the last 30 days (Yes/No)
💡 Click to see solution
WITH order_gaps AS (
    SELECT
        customer_id,
        order_date,
        total,
        order_date - LAG(order_date) OVER (
            PARTITION BY customer_id
            ORDER BY order_date
        ) AS days_since_prev
    FROM orders
),
customer_metrics AS (
    SELECT
        customer_id,
        COUNT(*) AS order_count,
        SUM(total) AS lifetime_value,
        MIN(order_date) AS first_order,
        MAX(order_date) AS last_order,
        AVG(days_since_prev) AS avg_days_between
    FROM order_gaps
    GROUP BY customer_id
)
SELECT
    c.name,
    cm.lifetime_value,
    cm.order_count,
    RANK() OVER (
        ORDER BY cm.lifetime_value DESC
    ) AS value_rank,
    cm.first_order,
    cm.last_order,
    ROUND(cm.avg_days_between, 1) AS avg_days_between,
    CASE
        WHEN cm.last_order >= CURRENT_DATE - INTERVAL '30 days'
        THEN 'Yes'
        ELSE 'No'
    END AS recent_customer
FROM customer_metrics cm
JOIN customers c ON c.customer_id = cm.customer_id
ORDER BY cm.lifetime_value DESC;

Key Takeaways

  • Subqueries let you nest queries for complex filtering and calculations
  • CTEs improve readability and can be recursive for hierarchical data
  • Window functions analyze rows without grouping them
  • PARTITION BY creates separate windows for different groups
  • Frame clauses control which rows are included in calculations
  • Combine techniques for powerful analytical queries