Cloud Platforms

AWS, GCP, and Azure, Storage, compute, databases, serverless, and cloud architecture

The Cloud: Infinite Infrastructure on Demand

Cloud computing revolutionized infrastructure. Instead of buying servers, racking them, and managing data centers, you provision resources with an API call and pay only for what you use. The big three, AWS (Amazon Web Services), GCP (Google Cloud Platform), and Azure (Microsoft Azure), dominate the market. This lesson focuses on AWS (the market leader with ~32% market share) while showing equivalent services in GCP and Azure. You'll learn storage, compute, databases, serverless, and how to architect cloud-native applications.

Cloud Fundamentals, Why Cloud?

Before diving into services, let's understand what cloud computing actually provides.

Speed & Agility
  • Provision resources in minutes (not months)
  • Experiment cheaply, fail fast
  • Global reach in 30+ regions
  • Deploy worldwide instantly
Cost Efficiency
  • Pay-as-you-go (no upfront costs)
  • Scale down = pay less
  • No hardware to maintain
  • Economies of scale
Elasticity
  • Auto-scale based on demand
  • Handle traffic spikes automatically
  • Scale to zero when idle
  • No capacity planning needed
Reliability
  • 99.99% SLAs for many services
  • Built-in redundancy
  • Automatic backups
  • Disaster recovery built-in

The Big Three

ProviderMarket ShareStrengthsBest For
AWS~32%Most mature, largest service catalog, huge ecosystemStartups, enterprises, general purpose
Azure~23%Enterprise integration, hybrid cloud, Windows/.NET workloadsEnterprises with Microsoft stack
GCP~10%Data analytics, ML/AI, Kubernetes, developer experienceData-heavy workloads, ML projects, startups

Storage Services, Objects, Blocks, and Files

Cloud storage comes in three flavors: object storage (files/blobs),block storage (virtual hard drives), and file storage (shared file systems).

Service TypeAWSGCPAzure
Object StorageS3Cloud StorageBlob Storage
Block StorageEBSPersistent DiskManaged Disks
File StorageEFSFilestoreAzure Files
Archive StorageS3 GlacierCloud Storage ArchiveArchive Blob Storage

AWS S3, Object Storage

S3 (Simple Storage Service) is AWS's flagship object storage. It's infinitely scalable, highly durable (99.999999999% - eleven 9s), and incredibly cheap.

# AWS CLI - S3 operations
# Create bucket
$ aws s3 mb s3://my-app-bucket

# Upload file
$ aws s3 cp image.jpg s3://my-app-bucket/images/

# Upload directory
$ aws s3 sync ./dist s3://my-app-bucket/website/

# Download file
$ aws s3 cp s3://my-app-bucket/data.csv ./

# List objects
$ aws s3 ls s3://my-app-bucket/

# Delete object
$ aws s3 rm s3://my-app-bucket/old-file.txt

# Make bucket public (careful!)
$ aws s3api put-bucket-policy --bucket my-app-bucket --policy file://policy.json

# Enable versioning
$ aws s3api put-bucket-versioning \
  --bucket my-app-bucket \
  --versioning-configuration Status=Enabled
# Python boto3 - S3 SDK
import boto3
from botocore.exceptions import ClientError

s3 = boto3.client('s3')

# Upload file
s3.upload_file('local.jpg', 'my-bucket', 'images/photo.jpg')

# Upload with metadata
s3.put_object(
    Bucket='my-bucket',
    Key='data.json',
    Body=json.dumps(data),
    ContentType='application/json',
    Metadata={'uploaded-by': 'user123'}
)

# Download file
s3.download_file('my-bucket', 'data.csv', 'local-data.csv')

# Generate presigned URL (temporary access)
url = s3.generate_presigned_url(
    'get_object',
    Params={'Bucket': 'my-bucket', 'Key': 'private.pdf'},
    ExpiresIn=3600  # 1 hour
)

# List objects with pagination
paginator = s3.get_paginator('list_objects_v2')
for page in paginator.paginate(Bucket='my-bucket', Prefix='logs/'):
    for obj in page.get('Contents', []):
        print(obj['Key'], obj['Size'])

# Delete multiple objects
s3.delete_objects(
    Bucket='my-bucket',
    Delete={
        'Objects': [
            {'Key': 'file1.txt'},
            {'Key': 'file2.txt'},
        ]
    }
)

S3 Storage Classes (Cost Optimization)

ClassUse CaseCostRetrieval Time
S3 StandardFrequently accessed data$$$$Instant
S3 Intelligent-TieringUnknown/changing access patternsAuto-optimizedInstant
S3 Standard-IAInfrequent access (backups)$$$Instant
S3 One Zone-IAInfrequent, non-critical$$Instant
S3 Glacier InstantArchive, instant retrieval$Instant
S3 Glacier FlexibleLong-term archive$Minutes to hours
S3 Glacier Deep ArchiveCompliance, rarely accessed¢12 hours

AWS EBS, Block Storage

EBS (Elastic Block Store) provides persistent block storage volumes for EC2 instances. Think of it as virtual hard drives.

# Create and attach EBS volume
# Create volume
$ aws ec2 create-volume \
  --availability-zone us-east-1a \
  --size 100 \
  --volume-type gp3 \
  --iops 3000 \
  --throughput 125

# Attach to instance
$ aws ec2 attach-volume \
  --volume-id vol-1234567890abcdef0 \
  --instance-id i-1234567890abcdef0 \
  --device /dev/sdf

# Create snapshot (backup)
$ aws ec2 create-snapshot \
  --volume-id vol-1234567890abcdef0 \
  --description "Backup before upgrade"

# Restore from snapshot
$ aws ec2 create-volume \
  --snapshot-id snap-1234567890abcdef0 \
  --availability-zone us-east-1a
EBS Volume Types:
gp3/gp2 (General Purpose SSD): Balanced price/performance, most workloads
io2/io1 (Provisioned IOPS SSD): High-performance databases, low latency
st1 (Throughput Optimized HDD): Big data, data warehouses
sc1 (Cold HDD): Infrequently accessed data

Compute Services, VMs, Containers, and Serverless

Compute is the backbone of cloud: virtual machines, containers, and serverless functions.

Service TypeAWSGCPAzure
Virtual MachinesEC2Compute EngineVirtual Machines
Container ServiceECS / EKSCloud Run / GKEContainer Instances / AKS
Serverless FunctionsLambdaCloud FunctionsAzure Functions
Serverless ContainersFargateCloud RunContainer Apps

AWS EC2, Virtual Machines

# Launch EC2 instance
# Via AWS CLI
$ aws ec2 run-instances \
  --image-id ami-0c55b159cbfafe1f0 \
  --instance-type t3.micro \
  --key-name my-key \
  --security-group-ids sg-1234567890 \
  --subnet-id subnet-1234567890 \
  --user-data file://install.sh \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=web-server}]'

# List instances
$ aws ec2 describe-instances \
  --filters "Name=tag:Environment,Values=production"

# Stop instance
$ aws ec2 stop-instances --instance-ids i-1234567890abcdef0

# Terminate instance
$ aws ec2 terminate-instances --instance-ids i-1234567890abcdef0
# Python boto3 - EC2
import boto3

ec2 = boto3.resource('ec2')

# Launch instance
instances = ec2.create_instances(
    ImageId='ami-0c55b159cbfafe1f0',
    MinCount=1,
    MaxCount=1,
    InstanceType='t3.micro',
    KeyName='my-key',
    SecurityGroupIds=['sg-1234567890'],
    SubnetId='subnet-1234567890',
    UserData='''#!/bin/bash
        yum update -y
        yum install -y httpd
        systemctl start httpd
        systemctl enable httpd
    ''',
    TagSpecifications=[
        {
            'ResourceType': 'instance',
            'Tags': [
                {'Key': 'Name', 'Value': 'web-server'},
                {'Key': 'Environment', 'Value': 'production'}
            ]
        }
    ]
)

# Wait for instance to be running
instance = instances[0]
instance.wait_until_running()

# Get public IP
instance.reload()
print(f"Instance running at {instance.public_ip_address}")

# Stop instance
instance.stop()

# Terminate instance
instance.terminate()

EC2 Instance Types

FamilyUse CaseExamples
T (Burstable)Low-cost, variable workloads, dev/testt3.micro, t3.medium
M (General Purpose)Balanced compute, memory, networkingm6i.large, m6i.xlarge
C (Compute Optimized)CPU-intensive, batch processing, ML inferencec6i.xlarge, c7g.2xlarge
R (Memory Optimized)In-memory databases, caches, analyticsr6i.xlarge, r7g.4xlarge
P/G (GPU)ML training, graphics renderingp4d.24xlarge, g5.xlarge

AWS Lambda, Serverless Functions

Lambda runs code without managing servers. Pay only for compute time used (billed per 100ms). Perfect for event-driven architectures.

# lambda_function.py
import json
import boto3
from datetime import datetime

s3 = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('users')

def lambda_handler(event, context):
    """
    Lambda handler function
    event: trigger data (API Gateway, S3 event, etc.)
    context: runtime information
    """

    # Example 1: API Gateway event
    if event.get('httpMethod'):
        body = json.loads(event['body'])

        # Process request
        user_id = body.get('user_id')
        user = table.get_item(Key={'id': user_id})

        return {
            'statusCode': 200,
            'headers': {'Content-Type': 'application/json'},
            'body': json.dumps(user.get('Item', {}))
        }

    # Example 2: S3 event (file uploaded)
    if event.get('Records'):
        for record in event['Records']:
            if record['eventName'].startswith('ObjectCreated'):
                bucket = record['s3']['bucket']['name']
                key = record['s3']['object']['key']

                # Process uploaded file
                print(f"Processing {key} from {bucket}")

                # Get file
                obj = s3.get_object(Bucket=bucket, Key=key)
                data = obj['Body'].read()

                # Do something with data...

        return {'statusCode': 200}

    # Example 3: Scheduled event (CloudWatch Events/EventBridge)
    if event.get('source') == 'aws.events':
        # Daily cleanup job
        print(f"Running cleanup at {datetime.now()}")
        # Cleanup logic...
        return {'statusCode': 200}

    return {
        'statusCode': 400,
        'body': json.dumps({'error': 'Unknown event type'})
    }
# Deploy Lambda with AWS SAM
# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  ApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: app.lambda_handler
      Runtime: python3.11
      CodeUri: ./src
      MemorySize: 512
      Timeout: 30
      Environment:
        Variables:
          TABLE_NAME: !Ref UsersTable
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /users/{id}
            Method: GET
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref UsersTable

  UsersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH

# Deploy
$ sam build
$ sam deploy --guided
Lambda Use Cases:
• API backends (with API Gateway)
• File processing (S3 uploads trigger Lambda)
• Scheduled tasks (cron jobs)
• Stream processing (Kinesis, DynamoDB Streams)
• Webhooks and integrations

Limits: 15-minute max execution, 10GB memory, 10GB ephemeral storage (512MB default)

Database Services, Relational, NoSQL, and Caching

Cloud providers offer managed databases: no patching, automated backups, high availability built-in.

Database TypeAWSGCPAzure
Relational (SQL)RDS (MySQL, PostgreSQL, etc.)Cloud SQLAzure SQL Database
NoSQL (Key-Value)DynamoDBFirestore, BigtableCosmos DB
Document DatabaseDocumentDBFirestoreCosmos DB
In-Memory CacheElastiCache (Redis, Memcached)MemorystoreAzure Cache for Redis
Data WarehouseRedshiftBigQuerySynapse Analytics

AWS RDS, Managed Relational Databases

# Create RDS instance (PostgreSQL)
$ aws rds create-db-instance \
  --db-instance-identifier myapp-db \
  --db-instance-class db.t4g.micro \
  --engine postgres \
  --engine-version 15.3 \
  --master-username admin \
  --master-user-password MySecurePassword123! \
  --allocated-storage 20 \
  --storage-type gp3 \
  --backup-retention-period 7 \
  --preferred-backup-window "03:00-04:00" \
  --multi-az \  # High availability
  --publicly-accessible false \
  --vpc-security-group-ids sg-1234567890 \
  --db-subnet-group-name my-db-subnet-group \
  --storage-encrypted \
  --enable-cloudwatch-logs-exports '["postgresql"]'

# Create read replica
$ aws rds create-db-instance-read-replica \
  --db-instance-identifier myapp-db-replica \
  --source-db-instance-identifier myapp-db \
  --db-instance-class db.t4g.micro

# Create snapshot
$ aws rds create-db-snapshot \
  --db-instance-identifier myapp-db \
  --db-snapshot-identifier myapp-db-snapshot-2024-01-20
# Connect to RDS from application
# Python with psycopg2
import psycopg2

# Get connection string from environment or Secrets Manager
conn = psycopg2.connect(
    host="myapp-db.c1234567890.us-east-1.rds.amazonaws.com",
    port=5432,
    database="myapp",
    user="admin",
    password="MySecurePassword123!"
)

cursor = conn.cursor()

# Query
cursor.execute("SELECT * FROM users WHERE email = %s", (email,))
user = cursor.fetchone()

# Insert
cursor.execute(
    "INSERT INTO users (name, email) VALUES (%s, %s)",
    (name, email)
)
conn.commit()

cursor.close()
conn.close()

AWS DynamoDB, Serverless NoSQL

DynamoDB is a fully managed NoSQL database. Single-digit millisecond latency, auto-scales to trillions of requests per day, pay per request.

# Python boto3 - DynamoDB
import boto3
from boto3.dynamodb.conditions import Key, Attr
from decimal import Decimal

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('users')

# Put item
table.put_item(
    Item={
        'user_id': '12345',
        'name': 'John Doe',
        'email': 'john@example.com',
        'age': 30,
        'tags': ['premium', 'verified'],
        'created_at': '2024-01-20T10:30:00Z'
    }
)

# Get item
response = table.get_item(
    Key={'user_id': '12345'}
)
user = response.get('Item')

# Update item
table.update_item(
    Key={'user_id': '12345'},
    UpdateExpression='SET age = :age, tags = list_append(tags, :tag)',
    ExpressionAttributeValues={
        ':age': 31,
        ':tag': ['active']
    }
)

# Query (requires partition key)
response = table.query(
    KeyConditionExpression=Key('user_id').eq('12345')
)

# Scan (expensive, avoid in production)
response = table.scan(
    FilterExpression=Attr('age').gt(25) & Attr('tags').contains('premium')
)

# Batch write
with table.batch_writer() as batch:
    for i in range(100):
        batch.put_item(
            Item={
                'user_id': f'user-{i}',
                'name': f'User {i}',
                'score': Decimal(str(i * 10))
            }
        )

# Delete item
table.delete_item(
    Key={'user_id': '12345'}
)
DynamoDB Key Concepts:
Partition Key: Required, used for data distribution
Sort Key: Optional, enables range queries
GSI (Global Secondary Index): Query on non-key attributes
LSI (Local Secondary Index): Alternative sort key
Billing: On-demand (pay per request) or provisioned capacity

Best for: High-scale apps, serverless, gaming leaderboards, IoT

Networking, VPCs, Load Balancers, and CDN

Cloud networking provides isolation, security, and global distribution.

ServiceAWSGCPAzure
Virtual NetworkVPCVPCVirtual Network
Load BalancerALB / NLB / ELBCloud Load BalancingLoad Balancer
CDNCloudFrontCloud CDNAzure CDN
DNSRoute 53Cloud DNSAzure DNS
API GatewayAPI GatewayAPI GatewayAPI Management

AWS VPC, Virtual Private Cloud

VPC Architecture (10.0.0.0/16)

Internet GatewayPublic AccessLoad BalancerAZ-A / 10.0.1.0/24Load BalancerAZ-B / 10.0.2.0/24Web ServerAZ-AWeb ServerAZ-BApp ServerPrivate 10.0.11.0/24App ServerPrivate 10.0.12.0/24RDS PrimaryAZ-ARDS ReplicaAZ-BReplication

Security: Security Groups, Network ACLs, VPC Flow Logs — NAT Gateway enables private subnet outbound access

Load Balancers

TypeLayerUse Case
ALB (Application)Layer 7 (HTTP/HTTPS)Web apps, microservices, path-based routing
NLB (Network)Layer 4 (TCP/UDP)Ultra-low latency, millions of requests/sec, static IP
ELB (Classic)Layer 4/7Legacy, use ALB/NLB instead
# Create Application Load Balancer
$ aws elbv2 create-load-balancer \
  --name my-alb \
  --subnets subnet-12345 subnet-67890 \
  --security-groups sg-12345 \
  --scheme internet-facing \
  --type application

# Create target group
$ aws elbv2 create-target-group \
  --name my-targets \
  --protocol HTTP \
  --port 80 \
  --vpc-id vpc-12345 \
  --health-check-path /health \
  --health-check-interval-seconds 30

# Register targets
$ aws elbv2 register-targets \
  --target-group-arn arn:aws:... \
  --targets Id=i-12345 Id=i-67890

# Create listener
$ aws elbv2 create-listener \
  --load-balancer-arn arn:aws:... \
  --protocol HTTP \
  --port 80 \
  --default-actions Type=forward,TargetGroupArn=arn:aws:...
CloudFront (CDN): Caches content at 600+ edge locations worldwide. Reduces latency, protects against DDoS, integrates with S3, ALB, custom origins.

Cloud Architecture Patterns

Common patterns for building cloud-native applications.

1. Three-Tier Web Application

Three-Tier Web Application

CloudFront + S3Presentation TierLoad BalancerApplication Tier (ALB)EC2 / ECSApp ServerEC2 / ECSApp ServerElastiCache RedisSession CacheRDS PrimaryPostgreSQLRDS ReplicaRead-onlyAPI CallsDB QueriesAsync

2. Serverless Architecture

Serverless Architecture

CloudFront + S3React / Vue / Angular SPAAPI GatewayCognito Auth, Rate LimitingLambdaGet UserLambdaCreate OrderLambdaProcess PaymentDynamoDBUsers | Orders | ProductsLambda TriggerProcess order eventsSQSAsync processing queueREST / GraphQLStreams

3. Microservices with ECS/EKS

Microservices with ECS / EKS

API Gateway / ALBUser ServiceProduct ServiceOrder ServicePayment ServiceEmail ServiceUsers DBProducts DBOrders DBPayments DBSESEmail
Each service:
• Independently deployable
• Own database (data isolation)
• Communicates via APIs/events
• Scales independently

Service Discovery: Route 53, AWS Cloud Map
Message Queue: SQS, SNS, EventBridge
Observability: CloudWatch, X-Ray

4. Event-Driven Architecture

Event-Driven Architecture

S3 BucketUser Upload TriggerS3 EventNotificationLambda 1Resize ImageLambda 2Extract MetadataLambda 3Generate ThumbnailS3OptimizedDynamoDBMetadataS3ThumbnailSNS Topic"File Ready"Notify Uservia EmailSearch IndexElasticsearchAnalyticsKinesisUpload

Benefits: Loose coupling, scalability, resilience

Architecture Principles:
• Design for failure (everything fails eventually)
• Implement elasticity (scale up/down automatically)
• Decouple components (loose coupling enables independent scaling)
• Think parallel (async processing, queues)
• Use managed services (less operational overhead)
• Implement security in depth (multiple layers)
• Cache aggressively (CloudFront, ElastiCache, DAX)

Cost Optimization

Cloud bills can spiral out of control. Here's how to optimize costs.

Right-Size Resources

Most instances are over-provisioned

  • Use AWS Compute Optimizer
  • Monitor actual usage (CloudWatch)
  • Start small, scale up if needed
  • Use burstable instances (t3) for variable loads
Reserved Instances / Savings Plans

Up to 72% savings for predictable workloads

  • 1-year or 3-year commitment
  • All upfront = maximum discount
  • Compute Savings Plans (flexible)
  • Good for steady-state workloads
Spot Instances

Up to 90% discount, can be interrupted

  • Batch processing, CI/CD
  • Stateless, fault-tolerant apps
  • Mix with on-demand for resilience
  • Use Spot Fleet for managed approach
Serverless First

Pay only for actual usage

  • Lambda: Pay per invocation
  • Fargate: No EC2 management
  • DynamoDB: Pay per request
  • S3: Pay for storage + requests

Cost Monitoring & Alerts

# Enable Cost Explorer
# View costs by service, region, tag

# Set up billing alerts
$ aws cloudwatch put-metric-alarm \
  --alarm-name billing-alert \
  --alarm-description "Alert when bill > $100" \
  --metric-name EstimatedCharges \
  --namespace AWS/Billing \
  --statistic Maximum \
  --period 21600 \
  --threshold 100 \
  --comparison-operator GreaterThanThreshold

# Tag everything for cost allocation
$ aws ec2 create-tags \
  --resources i-1234567890 \
  --tags Key=Project,Value=myapp Key=Environment,Value=prod

# Use AWS Budgets for tracking
# Set budget alerts at 50%, 80%, 100% of target
Cost Optimization Checklist:
□ Delete unused resources (old snapshots, unattached EBS volumes)
□ Enable S3 lifecycle policies (transition to Glacier)
□ Use Auto Scaling (scale down during off-hours)
□ Implement caching (CloudFront, ElastiCache)
□ Review CloudWatch logs retention (default: forever!)
□ Use AWS Cost Anomaly Detection
□ Regular cost reviews with stakeholders

Key Takeaways

  • Cloud Benefits: Speed, elasticity, cost efficiency, global reach, reliability
  • Storage: S3 (objects), EBS (blocks), EFS (files), choose based on access patterns
  • Compute: EC2 (VMs), Lambda (serverless), ECS/EKS (containers), right tool for job
  • Databases: RDS (relational), DynamoDB (NoSQL), ElastiCache (caching), managed services
  • Networking: VPC (isolation), ALB/NLB (load balancing), CloudFront (CDN)
  • Architecture Patterns: 3-tier, serverless, microservices, event-driven
  • GCP/Azure: Similar services with different names, concepts transfer
  • Cost Optimization: Right-size, reserved/spot instances, serverless, monitoring
  • Best Practices: Design for failure, use managed services, implement security in depth
  • Getting Started: Start with AWS (largest ecosystem), learn by doing, use free tier

Quick Service Reference

CategoryAWSWhen to Use
Object StorageS3Static websites, backups, data lakes, media files
Block StorageEBSEC2 instance storage, databases on EC2
VMsEC2Legacy apps, full OS control, specific software
Serverless FunctionsLambdaEvent-driven, APIs, data processing, automation
ContainersECS, EKS, FargateMicroservices, modernized apps, portable workloads
Relational DBRDS, AuroraTransactional apps, complex queries, ACID requirements
NoSQL DBDynamoDBHigh scale, key-value, document store, low latency
CachingElastiCacheSession store, database cache, reduce latency
Load BalancerALB, NLBDistribute traffic, high availability, auto-scaling
CDNCloudFrontStatic content, global distribution, low latency
Message QueueSQSDecouple services, async processing, buffering
Pub/SubSNSFan-out messaging, notifications, events