Compute with EC2
Elastic Compute Cloud is the foundational IaaS service in AWS, provides virtual machines on demand, fully configurable, available in seconds.
Introduction
EC2 gives you a virtual machine with your choice of CPU, memory, storage, and operating system. Unlike serverless (Lambda), you control the OS and everything running on it. EC2 is the right choice when you need persistent processes, specific software stacks, GPU access, or workloads that run continuously.
Quick Navigation
1. Instance Types
Families, sizes, and when to use each
2. AMIs and User Data
Boot images and bootstrapping scripts
3. Security Groups
Stateful firewall rules for instances
4. Launching Instances via CLI
Create, connect, and manage instances
5. Load Balancers and Auto Scaling
ALB, NLB, GWLB, and ASGs
6. Accessing Private Instances
Bastion hosts and SSM Session Manager
7. Placement Groups
Cluster, spread, and partition strategies
1. Instance Types
Instance types follow the pattern family.size. The family indicates the hardware optimization; the size controls the amount of CPU and memory.
General Purpose
t3, t4gWeb servers, dev/test, small databases. Burstable CPU - cheap but not for sustained load.
Compute Optimized
c6i, c7gBatch processing, gaming, video encoding. High CPU-to-memory ratio.
Memory Optimized
r6i, r7gIn-memory databases (Redis), large data caches, analytics with big datasets.
Accelerated (GPU)
p4, g5Machine learning training/inference, graphics rendering.
Storage Optimized
i4i, d3High-throughput OLAP, distributed file systems, data warehousing on-instance. These types include local NVMe instance store, ephemeral block storage physically attached to the host (see lesson 24).
Sizing tip: start with t3.micro (free tier eligible) for experiments. Use t3.medium or t3.large for dev workloads. Only move to a fixed-performance type (c, r, m families) once you hit sustained CPU throttling on a t-series.
2. AMIs and User Data
An Amazon Machine Image (AMI) is a snapshot of an OS and software configuration. When you launch an instance, you pick an AMI as the starting point. AWS provides official AMIs for Amazon Linux, Ubuntu, Windows Server, and others. You can also bake your own AMI from a running instance to create a golden image for your fleet.
User data is a script passed to the instance at launch. It runs once on first boot as root, making it ideal for bootstrapping software:
#!/bin/bash
# User data script - runs once on first boot as root
yum update -y
yum install -y python3 python3-pip
# Install and start a simple HTTP server
pip3 install fastapi uvicorn
cat > /home/ec2-user/app.py << 'EOF'
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def root():
return {"message": "Hello from EC2"}
EOF
# nohup: ignore SIGHUP so the process survives when the SSH session closes
# &: run in the background so the script continues
nohup uvicorn app:app --host 0.0.0.0 --port 80 &3. Security Groups
Security groups are stateful virtual firewalls attached to an instance (or other AWS resources). Stateful means if you allow inbound traffic on port 80, the response traffic is automatically allowed out without needing an explicit outbound rule.
- Rules are allow-only - there is no explicit deny, only allow
- By default all inbound is blocked, all outbound is allowed
- You can reference other security groups as sources (e.g., allow traffic from the ALB's security group only)
- Changes take effect immediately without restarting the instance
# Create a security group aws ec2 create-security-group \ --group-name web-sg \ --description "Allow HTTP and SSH" \ --vpc-id vpc-0123456789abcdef0 # Allow inbound SSH from your IP only aws ec2 authorize-security-group-ingress \ --group-id sg-0123456789abcdef0 \ --protocol tcp --port 22 \ --cidr 203.0.113.5/32 # Allow inbound HTTP from anywhere aws ec2 authorize-security-group-ingress \ --group-id sg-0123456789abcdef0 \ --protocol tcp --port 80 \ --cidr 0.0.0.0/0
4. Launching Instances via CLI
# Launch a t3.micro Amazon Linux 2023 instance
aws ec2 run-instances \
--image-id ami-0c02fb55956c7d316 \
--instance-type t3.micro \
--key-name my-key-pair \
--security-group-ids sg-0123456789abcdef0 \
--subnet-id subnet-0123456789abcdef0 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=web-server}]'
# List running instances
aws ec2 describe-instances \
--filters "Name=instance-state-name,Values=running" \
--query 'Reservations[].Instances[].[InstanceId,InstanceType,PublicIpAddress,Tags[?Key==`Name`].Value|[0]]' \
--output table
# Stop and terminate
aws ec2 stop-instances --instance-ids i-0123456789abcdef0
aws ec2 terminate-instances --instance-ids i-0123456789abcdef0Connecting via SSH:
- Create a key pair:
aws ec2 create-key-pair --key-name my-key-pair --query KeyMaterial --output text > my-key-pair.pem - Restrict permissions:
chmod 400 my-key-pair.pem - SSH into the instance:
ssh -i my-key-pair.pem ec2-user@<public-ip>
5. Load Balancers and Auto Scaling Groups
A load balancer sits in front of a group of backend servers and distributes incoming traffic across them. Its core purposes are:
- High availability - if one instance fails its health check, the LB stops sending it traffic automatically
- Horizontal scaling - new instances (added by an ASG) register with the LB and immediately start receiving traffic
- TLS termination - the LB handles the SSL/TLS handshake so backend instances receive plain HTTP, simplifying certificate management
- Routing - ALBs can inspect HTTP headers and paths to send different requests to different backend services
Load Balancer Types
AWS Elastic Load Balancing offers three active load balancer types. Each operates at a different network layer and serves a different purpose:
Application LB (ALB)
Layer 7 - HTTP/HTTPS- Path-based routing: /api/* to service A, /app/* to service B
- Host-based routing: api.example.com vs app.example.com
- WebSocket, HTTP/2, and gRPC support
- Native Lambda, ECS, and Cognito integration
Default choice for web apps and microservices.
Network LB (NLB)
Layer 4 - TCP/UDP/TLS- Millions of requests/sec with ultra-low latency
- Static Elastic IP per Availability Zone
- Preserves the client source IP address
- TLS passthrough or termination without HTTP overhead
High-throughput workloads or when clients need a static IP for whitelisting.
Gateway LB (GWLB)
Layer 3 - IP packets- Transparent bump-in-the-wire for virtual appliances
- Routes packets through third-party firewalls or IDS/IPS
- Uses GENEVE encapsulation to forward full IP packets
Inserting security appliances (firewalls, deep-packet inspection) into a VPC traffic path.
Target Groups and Listener Rules
A target group is a named pool of backend resources (EC2 instances, IP addresses, Lambda functions, or ECS tasks). Health checks run per group independently, so an unhealthy target stops receiving traffic without affecting other groups. ALB listener rules map incoming requests to a target group based on conditions evaluated in priority order:
Path-based routing
Match on the URL path:
/api/*- route to API target group/admin/*- route to admin target group/*(default) - route to frontend target group
Host-based routing
Match on the Host HTTP header:
api.example.com- API clusteradmin.example.com- admin cluster*.example.com- wildcard catch-all
ALB Path-Based Routing to Multiple Target Groups
The ALB evaluates listener rules in priority order and routes each request to the matching target group; the ASG registers new instances automatically
Auto Scaling Groups
An Auto Scaling Group (ASG) manages a fleet of EC2 instances across multiple Availability Zones. Define minimum, desired, and maximum counts; AWS adds or removes instances automatically and registers them with the ALB target group.
Launch Template
Specifies AMI, instance type, key pair, security groups, and user data. Versioned, so you can update and roll back without recreating the ASG.
Scaling Policies
Target tracking (maintain 50% avg CPU), step scaling (add 2 instances when CPU > 80%), or scheduled scaling (pre-scale before a known traffic spike).
Instance Refresh
Rolling replacement of running instances when you publish a new launch template version. Set a minimum healthy percentage (e.g., 90%) to maintain capacity during rollout.
6. Accessing Private Instances
The Problem
Production EC2 instances for application servers and databases live in private subnets with no public IP and no direct internet route. That is the correct security posture, but it creates a practical problem: how do you SSH in when you need to debug, run a migration, or inspect logs? Two answers exist: the classic Bastion Host and the modern SSM Session Manager.
What is a Bastion Host?
A bastion host (also called a jump server or jump box) is a hardened EC2 instance placed in a public subnet whose sole purpose is to act as the single, auditable entry point for SSH access to instances in private subnets. You SSH into the bastion first, then SSH onward from there to your private targets.
Bastion Host: Single Entry Point to Private Instances
Only the bastion is exposed on port 22; private instances allow SSH exclusively from the bastion's security group
Security Group Design
Bastion Security Group
Allow inbound SSH (:22) from your specific IP or corporate CIDR only, never 0.0.0.0/0. An open bastion is immediately probed by automated scanners worldwide.
Outbound: allow SSH to the VPC CIDR (e.g., 10.0.0.0/16) so the bastion can reach private instances. No other outbound rules are needed.
Private Instance Security Group
Allow inbound SSH from the bastion's security group ID, not from its IP. Using a security group reference means the rule remains correct even if the bastion is replaced with a new IP.
This is the pattern: source: sg-bastion-id. No CIDR. No manual IP updates when the bastion's Elastic IP changes.
# 1. Create the bastion security group - SSH from your IP only BASTION_SG=$(aws ec2 create-security-group \ --group-name bastion-sg \ --description "Bastion - SSH inbound" \ --vpc-id $VPC_ID \ --query GroupId --output text) # Never use 0.0.0.0/0 here - restrict to your IP or corporate CIDR aws ec2 authorize-security-group-ingress \ --group-id $BASTION_SG \ --protocol tcp --port 22 \ --cidr 203.0.113.5/32 # 2. Allow private instances to accept SSH from the bastion SG only # SG reference = no hardcoded CIDR, scales as bastion is replaced aws ec2 authorize-security-group-ingress \ --group-id $APP_SG \ --protocol tcp --port 22 \ --source-group $BASTION_SG # 3. Launch bastion in a public subnet with a static Elastic IP BASTION_ID=$(aws ec2 run-instances \ --image-id ami-0c02fb55956c7d316 \ --instance-type t3.micro \ --key-name my-key-pair \ --security-group-ids $BASTION_SG \ --subnet-id $PUBLIC_SUBNET_ID \ --query 'Instances[0].InstanceId' --output text) EIP=$(aws ec2 allocate-address --query AllocationId --output text) aws ec2 associate-address --instance-id $BASTION_ID --allocation-id $EIP # 4. Reach a private instance through the bastion using ProxyJump # -A forwards your local SSH agent; private key never touches the bastion ssh -A -J ec2-user@<bastion-public-ip> ec2-user@<private-instance-ip>
Operational Considerations
Patch the OS
The bastion is internet-facing and must be kept current. Enable automatic OS updates or replace it regularly with a freshly baked AMI.
Never store keys on the bastion
Use SSH agent forwarding (-A flag) so your local private key is used for the second hop. Storing keys on the bastion creates a single point of compromise.
Use an Elastic IP
Assign a static Elastic IP so your IP allowlist rule stays valid if the instance restarts. A changing public IP means constant SG rule updates.
Enable audit logging
Enable VPC Flow Logs and CloudTrail on the bastion's account. Consider shipping shell history to CloudWatch Logs for a full audit trail of commands run.
Minimize access
Only individuals who genuinely need server access should be on the allowlist. Review the SG inbound rules regularly. Remove IPs that no longer need access.
Stop it when not in use
In non-production environments, stop the bastion when not in use to reduce the attack surface and save on Elastic IP charges (EIPs on stopped instances are billed).
Modern Alternative: SSM Session Manager
AWS Systems Manager Session Manager provides interactive shell access to EC2 instances without opening any inbound ports, without SSH key pairs, and without a bastion host. The SSM Agent (pre-installed on Amazon Linux and available for all major OSes) opens an outbound HTTPS connection to the SSM endpoint. You connect through that channel, authenticated entirely by IAM.
Why it beats a bastion
- No port 22 open - zero inbound firewall rules needed
- No key pairs - access controlled by IAM policies
- Full audit trail - every session and command logged to CloudTrail and optionally to S3 or CloudWatch
- Works in private subnets - only requires outbound HTTPS to the SSM VPC endpoint
- Port forwarding - forward local ports to RDS, ElastiCache, or any private service without a bastion
Requirements
- SSM Agent installed and running on the instance
- IAM instance profile with
AmazonSSMManagedInstanceCorepolicy - Outbound HTTPS to
ssm.{region}.amazonaws.com(or a VPC endpoint for SSM) aws ssm start-sessionCLI plugin installed locally
# SSM Session Manager: shell access with no port 22, no key pairs, no bastion
# Requirements: SSM Agent on instance + IAM role with AmazonSSMManagedInstanceCore
# Open an interactive shell session
aws ssm start-session --target i-0123456789abcdef0
# Port-forward a private RDS port to your local machine (no bastion required)
aws ssm start-session \
--target i-0123456789abcdef0 \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["5432"],"localPortNumber":["5432"]}'
# Then connect your DB client to localhost:5432When to still use a bastion
SSM Session Manager is the preferred approach for new workloads. Bastions are still common when the target OS does not support the SSM Agent (older Windows Server, custom appliances), when corporate policy mandates SSH key-based access for compliance reasons, or when connecting to non-EC2 resources (on-premises servers behind a VPN) that the SSM Agent cannot reach.
7. Placement Groups
By default, AWS spreads your instances across hardware automatically. A placement group lets you override that behavior and tell AWS exactly how to position your instances relative to one another. There are three strategies, each optimizing for a different trade-off between performance and fault isolation:
Cluster
Pack instances onto hardware that is as close together as possible, ideally on the same physical rack within a single AZ.
Advantages:
- Lowest network latency between instances (single-digit microseconds)
- Highest inter-instance throughput (up to 100 Gbps with enhanced networking)
- Ideal for tightly coupled workloads that exchange large volumes of data
Limitations:
- All instances share fate with the rack: a hardware failure can take out the whole group
- Must be in a single AZ; cannot span AZs
- AWS may fail to launch if capacity on suitable hardware is unavailable
Use case: HPC, MPI jobs, distributed ML training (GPU instances), real-time analytics pipelines.
Spread
Place each instance on a distinct physical host with its own power supply and network switch. Maximum 7 running instances per AZ.
Advantages:
- Maximum fault isolation: no two instances share the same underlying hardware
- Can span multiple AZs within a region
- Suitable when losing any single instance would impact the application
Limitations:
- Hard limit of 7 running instances per AZ per placement group
- Not suitable for large fleets that need this guarantee
Use case: Small sets of critical instances: primary/standby databases, ZooKeeper quorum nodes, or any HA pair where simultaneous failure is unacceptable.
Partition
Divide instances into logical partitions (up to 7 per AZ). Instances within one partition may share a rack, but no two partitions share any hardware.
Advantages:
- Scales to hundreds of instances while still providing rack-level isolation
- AWS exposes partition metadata to the instance so the app can place replicas intelligently
- Can span multiple AZs
Limitations:
- No guarantee that instances within the same partition are on the same host
- Requires the application to be aware of partition topology to use it effectively
Use case: Large distributed systems that need replica placement awareness: Kafka, Cassandra, Hadoop HDFS, and other frameworks with built-in rack-awareness.
Quick Comparison
| Strategy | Hardware isolation | Max instances per AZ | Multi-AZ? | Network performance |
|---|---|---|---|---|
| Cluster | None (same rack) | No hard limit | No | Maximum (100 Gbps+) |
| Spread | Per-instance (distinct hosts) | 7 | Yes | Standard |
| Partition | Per-partition (distinct racks) | 7 partitions, unlimited instances | Yes | Standard |
# Create placement groups aws ec2 create-placement-group \ --group-name hpc-cluster \ --strategy cluster aws ec2 create-placement-group \ --group-name critical-spread \ --strategy spread aws ec2 create-placement-group \ --group-name kafka-partitions \ --strategy partition \ --partition-count 3 # Launch instances into a placement group aws ec2 run-instances \ --image-id ami-0c02fb55956c7d316 \ --instance-type c5n.18xlarge \ --placement "GroupName=hpc-cluster" \ --count 4 # Query which partition an instance landed on (from the instance itself) curl http://169.254.169.254/latest/meta-data/placement/partition-number
Capacity tip: Launching a cluster placement group with many instances at once is more likely to succeed than launching them incrementally, because AWS can reserve contiguous capacity in a single request. If you get an InsufficientInstanceCapacity error, stop all instances in the group and restart them together, or try a different AZ.
Key Takeaways
- Instance families - t for burstable, c for CPU, r for memory, p/g for GPU; start small and resize
- AMIs - the disk image your instance boots from; bake golden images for consistent fleet deployments
- User data - bootstrapping script that runs once on first boot as root, great for software installation
- Security groups - stateful, allow-only; reference other security groups to restrict traffic to specific services
- Load balancer types - ALB (Layer 7, HTTP routing rules) for web apps; NLB (Layer 4, static IP) for high-throughput; GWLB for inline security appliances
- Target groups and listener rules - ALB routes by path or host header to named backend pools; health checks run per group independently
- Auto Scaling Groups - min/desired/max fleet across AZs; scale on CloudWatch metrics; use instance refresh for rolling AMI deployments
- Bastion host - hardened EC2 in a public subnet acting as the single SSH entry point to private instances; lock down its SG to your IP only and use SSH agent forwarding so no private key is stored on it
- SSM Session Manager - preferred modern alternative: no port 22, no key pairs, IAM-controlled access, full CloudTrail audit trail, and supports port forwarding to private resources
- Placement groups - cluster (same rack, maximum throughput, HPC/ML), spread (distinct hosts, max 7 per AZ, HA pairs), partition (distinct racks, rack-aware distributed systems like Kafka and Cassandra)