Advanced Queries
Subqueries, CTEs, and window functions
Beyond Basic Queries
Once you master basic SELECT statements, joins, and aggregations, you're ready for advanced querying techniques. Subqueries let you nest queries inside queries. CTEs (Common Table Expressions) make complex queries readable and reusable. Window functions perform calculations across rows without grouping them. These tools unlock powerful analytical capabilities and help you write cleaner, more maintainable SQL.
Subqueries
A subquery is a query nested inside another query. Use it to break complex problems into smaller, logical steps.
Subquery in WHERE Clause
Find employees who earn more than the average salary.
SELECT name, salary FROM employees WHERE salary > (SELECT AVG(salary) FROM employees);
Alice | $85,000
Bob | $92,000
(Average was $70,000)
Subquery in SELECT Clause
Show each employee with the department's average salary.
SELECT
name,
salary,
(
SELECT AVG(salary)
FROM employees e2
WHERE e2.dept_id = e1.dept_id
) AS dept_avg
FROM employees e1;Alice | $85,000 | $78,500 (Engineering avg)
Bob | $72,000 | $78,500 (Engineering avg)
Carol | $65,000 | $67,000 (Sales avg)
Subquery in FROM Clause
Find departments with average salary above $75,000.
SELECT dept_name, avg_salary
FROM (
SELECT dept_id, AVG(salary) AS avg_salary
FROM employees
GROUP BY dept_id
) AS dept_averages
JOIN departments ON departments.id = dept_averages.dept_id
WHERE avg_salary > 75000;Engineering | $78,500
Product | $82,000
Subquery with IN
Find customers who have placed orders in the last 30 days.
SELECT name, email
FROM customers
WHERE customer_id IN (
SELECT DISTINCT customer_id
FROM orders
WHERE order_date >= CURRENT_DATE - INTERVAL '30 days'
);Alice Johnson | alice@example.com
Bob Smith | bob@example.com
(Only customers with recent orders)
Subquery with EXISTS
Find products that have never been ordered.
SELECT product_name, price
FROM products p
WHERE NOT EXISTS (
SELECT 1
FROM order_items oi
WHERE oi.product_id = p.product_id
);Vintage Camera | $899
Standing Desk | $549
(Products with 0 orders)
Common Table Expressions (CTEs)
CTEs (WITH clauses) create temporary named result sets that exist only for the duration of a query. They make complex queries more readable and maintainable.
Basic CTE
Find high-value customers (those who spent over $1,000).
WITH customer_totals AS (
SELECT
customer_id,
SUM(total) AS total_spent
FROM orders
GROUP BY customer_id
)
SELECT c.name, ct.total_spent
FROM customer_totals ct
JOIN customers c ON c.customer_id = ct.customer_id
WHERE ct.total_spent > 1000
ORDER BY ct.total_spent DESC;Alice Johnson | $2,450
Bob Smith | $1,850
Carol White | $1,320
Multiple CTEs
Compare department averages to company average.
WITH dept_averages AS (
SELECT
dept_id,
AVG(salary) AS avg_salary
FROM employees
GROUP BY dept_id
),
company_average AS (
SELECT AVG(salary) AS company_avg
FROM employees
)
SELECT
d.dept_name,
da.avg_salary,
ca.company_avg,
da.avg_salary - ca.company_avg AS difference
FROM dept_averages da
JOIN departments d ON d.dept_id = da.dept_id
CROSS JOIN company_average ca
ORDER BY difference DESC;Engineering | $78,500 | $70,000 | +$8,500
Product | $72,000 | $70,000 | +$2,000
Sales | $65,000 | $70,000 | -$5,000
Recursive CTE
Find an employee's entire management chain.
WITH RECURSIVE management_chain AS (
-- Base case: start with the employee
SELECT
employee_id,
name,
manager_id,
0 AS level
FROM employees
WHERE employee_id = 42
UNION ALL
-- Recursive case: find their manager
SELECT
e.employee_id,
e.name,
e.manager_id,
mc.level + 1
FROM employees e
JOIN management_chain mc ON e.employee_id = mc.manager_id
)
SELECT level, name
FROM management_chain
ORDER BY level;0 | John Doe (starting employee)
1 | Jane Manager
2 | Alice Director
3 | Bob CEO
Window Functions
Window functions perform calculations across rows that are related to the current row, without collapsing them into groups. Think of them as "looking through a window" at nearby rows.
ROW_NUMBER()
Assign a unique sequential number to each row.
SELECT
name,
salary,
ROW_NUMBER() OVER (ORDER BY salary DESC) AS rank
FROM employees;Bob | $92,000 | 1
Alice | $85,000 | 2
Carol | $72,000 | 3
David | $65,000 | 4
RANK() and DENSE_RANK()
Rank rows with ties. RANK skips numbers after ties, DENSE_RANK doesn't.
SELECT
name,
salary,
RANK() OVER (ORDER BY salary DESC) AS rank,
DENSE_RANK() OVER (ORDER BY salary DESC) AS dense_rank
FROM employees;Bob | $92,000 | 1 | 1
Alice | $85,000 | 2 | 2
Carol | $72,000 | 3 | 3
David | $72,000 | 3 | 3 (tied)
Eve | $65,000 | 5 | 4 (RANK skipped 4)
PARTITION BY
Rank employees within each department separately.
SELECT
dept_name,
name,
salary,
RANK() OVER (PARTITION BY dept_id ORDER BY salary DESC) AS dept_rank
FROM employees
JOIN departments USING (dept_id);Engineering | Alice | $85,000 | 1
Engineering | Bob | $72,000 | 2
Sales | Carol | $65,000 | 1
Sales | David | $62,000 | 2
(Rankings restart for each department)
Running Totals with SUM()
Calculate cumulative sales over time.
SELECT
order_date,
total,
SUM(total) OVER (ORDER BY order_date) AS running_total
FROM orders
ORDER BY order_date;2024-01-01 | $100 | $100
2024-01-02 | $150 | $250
2024-01-03 | $200 | $450
2024-01-04 | $175 | $625
Moving Average
Calculate 3-day moving average of sales.
SELECT
order_date,
total,
AVG(total) OVER (
ORDER BY order_date
ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
) AS moving_avg_3day
FROM orders
ORDER BY order_date;2024-01-01 | $100 | $100 (only 1 day)
2024-01-02 | $150 | $125 (2 days)
2024-01-03 | $200 | $150 (3 days: 100+150+200)
2024-01-04 | $175 | $175 (3 days: 150+200+175)
LAG() and LEAD()
Access previous or next row's value.
SELECT
order_date,
total,
LAG(total) OVER (ORDER BY order_date) AS prev_day,
total - LAG(total) OVER (ORDER BY order_date) AS change
FROM orders
ORDER BY order_date;2024-01-01 | $100 | NULL | NULL
2024-01-02 | $150 | $100 | +$50
2024-01-03 | $200 | $150 | +$50
2024-01-04 | $175 | $200 | -$25
NTILE()
Divide rows into N equal groups (quartiles, deciles, etc.).
SELECT
name,
salary,
NTILE(4) OVER (ORDER BY salary DESC) AS quartile
FROM employees;Bob | $92,000 | 1 (top 25%)
Alice | $85,000 | 1
Carol | $72,000 | 2
David | $65,000 | 3
Eve | $58,000 | 4 (bottom 25%)
Window Frame Clauses
Frame clauses define which rows the window function considers. They let you control the "window" size.
ROWS
Physical rows relative to current
ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
RANGE
Logical range based on values
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
Common Frame Specifications
-- All rows from start to current ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW -- Current and next row ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING -- Previous 3, current, and next 3 rows ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING -- All rows in partition ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
Real-World Analytics Examples
Top N per Category
Find the top 3 selling products in each category.
WITH ranked_products AS (
SELECT
category_name,
product_name,
total_sales,
ROW_NUMBER() OVER (
PARTITION BY category_id
ORDER BY total_sales DESC
) AS rank
FROM products
JOIN categories USING (category_id)
)
SELECT category_name, product_name, total_sales
FROM ranked_products
WHERE rank <= 3
ORDER BY category_name, rank;Electronics | Laptop Pro | $125,000
Electronics | Wireless Mouse | $45,000
Electronics | USB-C Cable | $23,000
Furniture | Office Chair | $89,000
Furniture | Standing Desk | $67,000
Furniture | Monitor Arm | $34,000
Year-over-Year Growth
Compare monthly sales to the same month last year.
SELECT
month,
year,
monthly_sales,
LAG(monthly_sales, 12) OVER (ORDER BY year, month) AS prev_year_sales,
ROUND(
(monthly_sales - LAG(monthly_sales, 12) OVER (ORDER BY year, month))
/ LAG(monthly_sales, 12) OVER (ORDER BY year, month) * 100, 1
) AS yoy_growth_pct
FROM monthly_sales_summary
ORDER BY year, month;Jan | 2023 | $50,000 | NULL | NULL
Jan | 2024 | $58,000 | $50,000 | +16.0%
Feb | 2024 | $62,000 | $52,000 | +19.2%
Customer Lifetime Value Segments
Segment customers by their total spending.
WITH customer_spending AS (
SELECT
customer_id,
SUM(total) AS lifetime_value,
NTILE(3) OVER (ORDER BY SUM(total) DESC) AS value_segment
FROM orders
GROUP BY customer_id
)
SELECT CASE
value_segment
WHEN 1 THEN 'High Value'
WHEN 2 THEN 'Medium Value'
WHEN 3 THEN 'Low Value'
END AS segment,
COUNT(*) AS customer_count,
ROUND(AVG(lifetime_value), 2) AS avg_ltv,
ROUND(MIN(lifetime_value), 2) AS min_ltv,
ROUND(MAX(lifetime_value), 2) AS max_ltv
FROM customer_spending
GROUP BY value_segment
ORDER BY value_segment;High Value | 334 | $2,450.50 | $1,200 | $5,890
Medium Value | 333 | $675.25 | $450 | $1,199
Low Value | 333 | $185.75 | $50 | $449
Cohort Analysis
Analyze customer retention by signup cohort.
WITH first_purchase AS (
SELECT
customer_id,
DATE_TRUNC('month', MIN(order_date)) AS cohort_month
FROM orders
GROUP BY customer_id
),
cohort_activity AS (
SELECT
fp.cohort_month,
DATE_TRUNC('month', o.order_date) AS activity_month,
COUNT(DISTINCT o.customer_id) AS active_customers
FROM first_purchase fp
JOIN orders o ON o.customer_id = fp.customer_id
GROUP BY fp.cohort_month, DATE_TRUNC('month', o.order_date)
)
SELECT
cohort_month,
activity_month,
active_customers,
FIRST_VALUE(active_customers) OVER (
PARTITION BY cohort_month
ORDER BY activity_month
) AS cohort_size,
ROUND(
active_customers::NUMERIC /
FIRST_VALUE(active_customers) OVER (
PARTITION BY cohort_month
ORDER BY activity_month
) * 100,
1
) AS retention_pct
FROM cohort_activity
ORDER BY cohort_month, activity_month;2024-01 | 2024-01 | 100 | 100 | 100.0%
2024-01 | 2024-02 | 65 | 100 | 65.0%
2024-01 | 2024-03 | 52 | 100 | 52.0%
(Shows how Jan 2024 cohort retained over time)
Performance Tips
✅ Index Key Columns
Add indexes on columns used in subquery WHERE clauses and window function ORDER BY clauses for better performance.
✅ Limit Subquery Results
Filter early in subqueries to reduce the amount of data being processed. Don't fetch everything then filter.
✅ Use CTEs for Readability
CTEs make complex queries maintainable and are often optimized as well as equivalent subqueries by modern databases.
✅ Minimize Window Frames
Smaller window frames (e.g., 3 PRECEDING instead of UNBOUNDED PRECEDING) perform better for large datasets.
❌ Avoid Correlated Subqueries
Subqueries that reference outer query columns run once per row and can be very slow. Use JOINs or window functions instead.
❌ Don't Nest Too Deeply
Multiple levels of nested subqueries become hard to read and debug. Break them into CTEs instead.
Common Query Patterns
Pattern 1: Find Duplicates
SELECT email, COUNT(*) AS duplicate_count FROM users GROUP BY email HAVING COUNT(*) > 1;
john@example.com | 3
jane@example.com | 2
Pattern 2: Running Difference
SELECT
date,
inventory_count,
inventory_count - LAG(inventory_count) OVER (ORDER BY date) AS daily_change
FROM inventory_snapshots
ORDER BY date;2024-01-01 | 1000 | NULL
2024-01-02 | 985 | -15
2024-01-03 | 1020 | +35
Pattern 3: Percent of Total
SELECT
product_name,
sales,
ROUND(
sales * 100.0 / SUM(sales) OVER (),
2
) AS pct_of_total
FROM product_sales
ORDER BY sales DESC;Laptop | $50,000 | 35.71%
Mouse | $40,000 | 28.57%
Monitor | $30,000 | 21.43%
Keyboard | $20,000 | 14.29%
Pattern 4: First/Last in Group
WITH ranked_orders AS (
SELECT
customer_id,
order_date,
total,
ROW_NUMBER() OVER (
PARTITION BY customer_id
ORDER BY order_date DESC
) AS recency_rank
FROM orders
)
SELECT customer_id, order_date, total
FROM ranked_orders
WHERE recency_rank = 1;42 | 2024-03-15 | $250
43 | 2024-03-14 | $180
44 | 2024-03-16 | $320
When to Use What
| Technique | Use When | Example |
|---|---|---|
| Subquery in WHERE | Filtering based on aggregated data | Employees above average salary |
| Subquery in SELECT | Adding calculated column from related data | Show department average with each employee |
| CTE | Complex queries, multiple references, readability | Multi-step analysis with intermediate results |
| Recursive CTE | Hierarchical or graph data | Org charts, bill of materials, category trees |
| ROW_NUMBER() | Unique sequential numbering | Top N per group, pagination |
| RANK() | Ranking with gaps for ties | Leaderboards, competition rankings |
| Running Total | Cumulative calculations | Year-to-date sales, cumulative metrics |
| LAG/LEAD | Compare to previous/next row | Period-over-period growth, changes |
Practice Challenge
Challenge: Customer Purchase Analysis
Write a query that shows for each customer:
- Customer name and total lifetime value
- Number of orders they've placed
- Their rank by lifetime value (highest = 1)
- Date of their first and most recent order
- Average days between orders
- Whether they've ordered in the last 30 days (Yes/No)
💡 Click to see solution
WITH order_gaps AS (
SELECT
customer_id,
order_date,
total,
order_date - LAG(order_date) OVER (
PARTITION BY customer_id
ORDER BY order_date
) AS days_since_prev
FROM orders
),
customer_metrics AS (
SELECT
customer_id,
COUNT(*) AS order_count,
SUM(total) AS lifetime_value,
MIN(order_date) AS first_order,
MAX(order_date) AS last_order,
AVG(days_since_prev) AS avg_days_between
FROM order_gaps
GROUP BY customer_id
)
SELECT
c.name,
cm.lifetime_value,
cm.order_count,
RANK() OVER (
ORDER BY cm.lifetime_value DESC
) AS value_rank,
cm.first_order,
cm.last_order,
ROUND(cm.avg_days_between, 1) AS avg_days_between,
CASE
WHEN cm.last_order >= CURRENT_DATE - INTERVAL '30 days'
THEN 'Yes'
ELSE 'No'
END AS recent_customer
FROM customer_metrics cm
JOIN customers c ON c.customer_id = cm.customer_id
ORDER BY cm.lifetime_value DESC;Key Takeaways
- Subqueries let you nest queries for complex filtering and calculations
- CTEs improve readability and can be recursive for hierarchical data
- Window functions analyze rows without grouping them
- PARTITION BY creates separate windows for different groups
- Frame clauses control which rows are included in calculations
- Combine techniques for powerful analytical queries