Infrastructure as Code

Declarative vs programmatic infrastructure, Terraform, CloudFormation, AWS CDK, Pulumi, and Ansible

Infrastructure as Code: Code Your Cloud

Clicking through cloud consoles doesn't scale. Infrastructure as Code (IaC) treats infrastructure like software: versioned, tested, reviewed, and automated. But not all IaC is created equal. Declarative tools like Terraform and CloudFormation use configuration files to describe desired state. Programmatic tools like AWS CDK and Pulumi use real programming languages, giving you the full power of code: loops, conditionals, functions, type safety, and IDE support. This lesson covers both approaches and when to use each.

IaC Fundamentals, Why and What

Infrastructure as Code means defining your infrastructure in code files instead of manual configuration.

❌ Manual Infrastructure

Click through AWS console
No version control
Hard to replicate environments
Tribal knowledge, documentation outdated
Drift between dev/staging/prod
Disaster recovery = panic
Can't review infrastructure changes

✅ Infrastructure as Code

Define infrastructure in code files
Version control (Git)
Identical environments via code
Self-documenting
Environment parity guaranteed
Disaster recovery = re-run code
Code reviews for infra changes

Key Benefits

Reproducibility

Create identical environments on demand
Dev = Staging = Production (except config)
Spin up test environment, destroy after tests

Version Control

Every change tracked in Git
Who changed what, when, and why
Easy rollback: git revert
Code review for infrastructure

Automation

CI/CD for infrastructure
Automated testing (linting, dry-run)
Scheduled updates (patch Tuesday)
Self-service for developers

Documentation

Code is the documentation
Always up-to-date (unlike wiki)
Onboarding: read the code
Diagram generation from code

Cost Management

Destroy environments when not needed
Standardize instance types
Enforce tagging for cost allocation
Prevent over-provisioning

Declarative vs Programmatic IaC

Aspect	Declarative (Terraform, CloudFormation)	Programmatic (AWS CDK, Pulumi)
Language	DSL (HCL, YAML, JSON)	Real programming languages (Python, TypeScript, Go)
Approach	"What" you want (desired state)	"How" to create it (imperative + declarative)
Abstraction	Limited (config-based)	High (functions, classes, packages)
Reusability	Modules (somewhat clunky)	Full OOP, npm/pip packages
IDE Support	Limited autocomplete	Full IDE features (IntelliSense, refactoring)
Testing	Limited unit testing	Full testing frameworks (Jest, pytest)
Learning Curve	New DSL to learn	Use languages you already know
Flexibility	Constrained by DSL	Full programming power (loops, conditionals, etc.)

Terraform, Declarative Multi-Cloud IaC

Terraform uses HashiCorp Configuration Language (HCL) to declare infrastructure. It's cloud-agnostic, has a massive provider ecosystem, and is the industry standard for declarative IaC.

Basic Terraform Structure

# main.tf

# Provider configuration
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "prod/terraform.tfstate"
    region = "us-east-1"
  }
}

provider "aws" {
  region = var.aws_region
}

# VPC
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "${var.project_name}-vpc"
    Environment = var.environment
  }
}

# Subnet
resource "aws_subnet" "public" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = var.availability_zones[count.index]

  map_public_ip_on_launch = true

  tags = {
    Name = "${var.project_name}-public-${var.availability_zones[count.index]}"
  }
}

# Security Group
resource "aws_security_group" "web" {
  name        = "${var.project_name}-web-sg"
  description = "Security group for web servers"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# EC2 Instance
resource "aws_instance" "web" {
  count                  = var.instance_count
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  subnet_id              = aws_subnet.public[count.index % length(aws_subnet.public)].id
  vpc_security_group_ids = [aws_security_group.web.id]

  user_data = file("user-data.sh")

  tags = {
    Name = "${var.project_name}-web-${count.index + 1}"
  }
}

# Data source for latest Ubuntu AMI
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}

# variables.tf

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "project_name" {
  description = "Project name"
  type        = string
}

variable "environment" {
  description = "Environment (dev, staging, prod)"
  type        = string
}

variable "vpc_cidr" {
  description = "VPC CIDR block"
  type        = string
  default     = "10.0.0.0/16"
}

variable "availability_zones" {
  description = "Availability zones"
  type        = list(string)
  default     = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

variable "instance_count" {
  description = "Number of instances"
  type        = number
  default     = 2
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"
}

# outputs.tf

output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

output "instance_ips" {
  description = "Public IPs of instances"
  value       = aws_instance.web[*].public_ip
}

output "security_group_id" {
  description = "Security group ID"
  value       = aws_security_group.web.id
}

Terraform Workflow

# Initialize (download providers)
$ terraform init

# Format code
$ terraform fmt

# Validate configuration
$ terraform validate

# Plan (dry-run, shows what will change)
$ terraform plan -var-file="prod.tfvars" -out=plan.out

# Apply changes
$ terraform apply plan.out

# Show current state
$ terraform show

# List resources
$ terraform state list

# Import existing resource
$ terraform import aws_instance.web i-1234567890abcdef0

# Destroy everything
$ terraform destroy -var-file="prod.tfvars"

Terraform Modules (Reusability)

# modules/vpc/main.tf

variable "vpc_cidr" { type = string }
variable "name" { type = string }

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
  tags = { Name = var.name }
}

output "vpc_id" {
  value = aws_vpc.this.id
}

output "vpc_cidr" {
  value = aws_vpc.this.cidr_block
}

# main.tf (using module)

module "vpc" {
  source   = "./modules/vpc"
  vpc_cidr = "10.0.0.0/16"
  name     = "production-vpc"
}

# Use module outputs
resource "aws_subnet" "example" {
  vpc_id     = module.vpc.vpc_id
  cidr_block = "10.0.1.0/24"
}

# Or use public modules from registry
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.1.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]

  enable_nat_gateway = true
  enable_vpn_gateway = false
}

Terraform Strengths:
• Cloud-agnostic (AWS, GCP, Azure, 3000+ providers)
• Huge community and module registry
• Mature, battle-tested
• Good state management

Limitations:
• HCL has limited expressiveness (no real loops, functions)
• Module reusability is clunky
• Testing is limited
• Refactoring is painful

AWS CDK, Programmatic Infrastructure with TypeScript/Python

AWS CDK (Cloud Development Kit) lets you define AWS infrastructure using real programming languages. It synthesizes to CloudFormation but gives you the full power of code: classes, functions, loops, conditionals, type safety, and IDE support.

CDK with TypeScript

# lib/my-stack.ts

import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns';
import * as rds from 'aws-cdk-lib/aws-rds';
import { Construct } from 'constructs';

export interface MyStackProps extends cdk.StackProps {
  environment: 'dev' | 'staging' | 'prod';
  instanceCount: number;
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    // VPC with public and private subnets
    const vpc = new ec2.Vpc(this, 'VPC', {
      maxAzs: 3,
      natGateways: props.environment === 'prod' ? 2 : 1,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'public',
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 24,
          name: 'private',
          subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
        },
        {
          cidrMask: 28,
          name: 'isolated',
          subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
        },
      ],
    });

    // ECS Cluster
    const cluster = new ecs.Cluster(this, 'Cluster', {
      vpc,
      containerInsights: props.environment === 'prod',
    });

    // Fargate Service with ALB
    const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(
      this,
      'FargateService',
      {
        cluster,
        cpu: this.getCpuForEnvironment(props.environment),
        memoryLimitMiB: this.getMemoryForEnvironment(props.environment),
        desiredCount: props.instanceCount,
        taskImageOptions: {
          image: ecs.ContainerImage.fromRegistry('nginx:latest'),
          environment: {
            ENVIRONMENT: props.environment,
          },
        },
        publicLoadBalancer: true,
      }
    );

    // Auto-scaling based on CPU
    const scaling = fargateService.service.autoScaleTaskCount({
      minCapacity: props.instanceCount,
      maxCapacity: props.instanceCount * 3,
    });

    scaling.scaleOnCpuUtilization('CpuScaling', {
      targetUtilizationPercent: 70,
      scaleInCooldown: cdk.Duration.seconds(60),
      scaleOutCooldown: cdk.Duration.seconds(60),
    });

    // RDS Database (only in staging/prod)
    if (props.environment !== 'dev') {
      const database = new rds.DatabaseInstance(this, 'Database', {
        engine: rds.DatabaseInstanceEngine.postgres({
          version: rds.PostgresEngineVersion.VER_15,
        }),
        vpc,
        vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_ISOLATED },
        instanceType: ec2.InstanceType.of(
          ec2.InstanceClass.T4G,
          props.environment === 'prod' ? ec2.InstanceSize.MEDIUM : ec2.InstanceSize.SMALL
        ),
        multiAz: props.environment === 'prod',
        allocatedStorage: props.environment === 'prod' ? 100 : 20,
        maxAllocatedStorage: props.environment === 'prod' ? 200 : 50,
        deletionProtection: props.environment === 'prod',
        backupRetention: cdk.Duration.days(props.environment === 'prod' ? 7 : 1),
      });

      // Allow connections from Fargate service
      database.connections.allowFrom(
        fargateService.service,
        ec2.Port.tcp(5432)
      );
    }

    // Outputs
    new cdk.CfnOutput(this, 'LoadBalancerDNS', {
      value: fargateService.loadBalancer.loadBalancerDnsName,
      description: 'DNS name of the load balancer',
    });
  }

  // Helper methods - this is real programming!
  private getCpuForEnvironment(env: string): number {
    const cpuMap = { dev: 256, staging: 512, prod: 1024 };
    return cpuMap[env as keyof typeof cpuMap];
  }

  private getMemoryForEnvironment(env: string): number {
    const memoryMap = { dev: 512, staging: 1024, prod: 2048 };
    return memoryMap[env as keyof typeof memoryMap];
  }
}

# bin/app.ts

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { MyStack } from '../lib/my-stack';

const app = new cdk.App();

// Create multiple environments programmatically
const environments = [
  { name: 'dev', instanceCount: 1 },
  { name: 'staging', instanceCount: 2 },
  { name: 'prod', instanceCount: 3 },
] as const;

for (const env of environments) {
  new MyStack(app, `MyStack-${env.name}`, {
    environment: env.name,
    instanceCount: env.instanceCount,
    env: {
      account: process.env.CDK_DEFAULT_ACCOUNT,
      region: process.env.CDK_DEFAULT_REGION,
    },
    tags: {
      Environment: env.name,
      ManagedBy: 'CDK',
    },
  });
}

CDK with Python

# app.py

from aws_cdk import (
    Stack,
    aws_s3 as s3,
    aws_lambda as lambda_,
    aws_apigateway as apigw,
    aws_dynamodb as dynamodb,
    RemovalPolicy,
    Duration,
)
from constructs import Construct

class ServerlessApiStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        # DynamoDB Table
        table = dynamodb.Table(
            self, "ItemsTable",
            partition_key=dynamodb.Attribute(
                name="id",
                type=dynamodb.AttributeType.STRING
            ),
            billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST,
            removal_policy=RemovalPolicy.DESTROY,  # For demo only
        )

        # Lambda Function
        handler = lambda_.Function(
            self, "ApiHandler",
            runtime=lambda_.Runtime.PYTHON_3_11,
            code=lambda_.Code.from_asset("lambda"),
            handler="handler.main",
            environment={
                "TABLE_NAME": table.table_name,
            },
            timeout=Duration.seconds(30),
        )

        # Grant Lambda permissions to DynamoDB
        table.grant_read_write_data(handler)

        # API Gateway
        api = apigw.RestApi(
            self, "ItemsApi",
            rest_api_name="Items Service",
            description="Serverless API with DynamoDB backend",
        )

        # Add endpoints
        items = api.root.add_resource("items")
        items.add_method("GET", apigw.LambdaIntegration(handler))
        items.add_method("POST", apigw.LambdaIntegration(handler))

        item = items.add_resource("{id}")
        item.add_method("GET", apigw.LambdaIntegration(handler))
        item.add_method("PUT", apigw.LambdaIntegration(handler))
        item.add_method("DELETE", apigw.LambdaIntegration(handler))

# Example of creating custom constructs (reusable components)
class WebsiteConstruct(Construct):
    """Reusable construct for static website"""

    def __init__(self, scope: Construct, id: str, domain_name: str):
        super().__init__(scope, id)

        # S3 bucket for website
        self.bucket = s3.Bucket(
            self, "WebsiteBucket",
            website_index_document="index.html",
            public_read_access=True,
            removal_policy=RemovalPolicy.DESTROY,
            auto_delete_objects=True,
        )

# Use the custom construct
app = cdk.App()
stack = Stack(app, "MyWebsiteStack")
website = WebsiteConstruct(stack, "Website", domain_name="example.com")

CDK Workflow

# Install CDK
$ npm install -g aws-cdk

# Create new project
$ cdk init app --language typescript
# or: --language python

# Install dependencies
$ npm install

# Synthesize CloudFormation template
$ cdk synth

# Diff against deployed stack
$ cdk diff

# Deploy
$ cdk deploy

# Deploy all stacks
$ cdk deploy --all

# Deploy specific stack
$ cdk deploy MyStack-prod

# Destroy
$ cdk destroy

# List all stacks
$ cdk list

CDK Advantages:
• Real programming languages (TypeScript, Python, Java, C#, Go)
• Full IDE support (autocomplete, type checking, refactoring)
• Reusable constructs (like npm packages)
• Unit testing with familiar frameworks
• Higher-level abstractions (L2/L3 constructs)
• Imperative + declarative programming

Considerations:
• AWS-only (not multi-cloud)
• Synthesizes to CloudFormation (inherits CF limits)
• Learning curve if new to programming

Pulumi, Multi-Cloud Programmatic IaC

Pulumi is like CDK but multi-cloud. Define infrastructure in TypeScript, Python, Go, C#, or Java. It manages state directly (not through CloudFormation) and supports AWS, GCP, Azure, Kubernetes, and more.

Pulumi with TypeScript

# index.ts

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Configuration
const config = new pulumi.Config();
const instanceCount = config.getNumber("instanceCount") || 2;
const environment = pulumi.getStack(); // dev, staging, prod

// VPC
const vpc = new awsx.ec2.Vpc("my-vpc", {
  numberOfAvailabilityZones: 3,
  natGateways: {
    strategy: environment === "prod" ? "OnePerAz" : "Single",
  },
  tags: {
    Environment: environment,
  },
});

// ECS Cluster
const cluster = new aws.ecs.Cluster("cluster", {
  vpcId: vpc.vpcId,
  tags: { Environment: environment },
});

// Application Load Balancer
const alb = new awsx.lb.ApplicationLoadBalancer("app-lb", {
  vpc: vpc,
  external: true,
});

// Fargate Service
const service = new awsx.ecs.FargateService("app-service", {
  cluster: cluster.arn,
  desiredCount: instanceCount,
  taskDefinitionArgs: {
    container: {
      image: "nginx:latest",
      cpu: getCpuForEnvironment(environment),
      memory: getMemoryForEnvironment(environment),
      essential: true,
      portMappings: [{
        containerPort: 80,
        targetGroup: alb.defaultTargetGroup,
      }],
    },
  },
  networkConfiguration: {
    subnets: vpc.privateSubnetIds,
    securityGroups: [cluster.securityGroups[0].id],
  },
});

// RDS Database (conditional)
let database: aws.rds.Instance | undefined;
if (environment !== "dev") {
  database = new aws.rds.Instance("postgres", {
    engine: "postgres",
    engineVersion: "15.3",
    instanceClass: environment === "prod" ? "db.t4g.medium" : "db.t4g.small",
    allocatedStorage: environment === "prod" ? 100 : 20,
    dbSubnetGroupName: vpc.privateSubnetIds.apply(subnets =>
      new aws.rds.SubnetGroup("db-subnet", {
        subnetIds: subnets,
      }).name
    ),
    multiAz: environment === "prod",
    backupRetentionPeriod: environment === "prod" ? 7 : 1,
    skipFinalSnapshot: environment !== "prod",
    vpcSecurityGroupIds: [cluster.securityGroups[0].id],
  });
}

// Outputs
export const vpcId = vpc.vpcId;
export const albDns = alb.loadBalancer.dnsName;
export const dbEndpoint = database?.endpoint;

// Helper functions
function getCpuForEnvironment(env: string): number {
  const cpuMap: Record<string, number> = {
    dev: 256,
    staging: 512,
    prod: 1024,
  };
  return cpuMap[env] || 256;
}

function getMemoryForEnvironment(env: string): number {
  const memoryMap: Record<string, number> = {
    dev: 512,
    staging: 1024,
    prod: 2048,
  };
  return memoryMap[env] || 512;
}

Pulumi with Python

# __main__.py

import pulumi
import pulumi_aws as aws
import pulumi_kubernetes as k8s
from pulumi_aws import ec2, eks

# Configuration
config = pulumi.Config()
node_count = config.get_int("node_count", 3)
node_type = config.get("node_type", "t3.medium")

# VPC for EKS
vpc = ec2.Vpc(
    "eks-vpc",
    cidr_block="10.0.0.0/16",
    enable_dns_hostnames=True,
    enable_dns_support=True,
)

# Subnets
subnet_ids = []
for i, az in enumerate(["us-east-1a", "us-east-1b", "us-east-1c"]):
    subnet = ec2.Subnet(
        f"eks-subnet-{i}",
        vpc_id=vpc.id,
        cidr_block=f"10.0.{i}.0/24",
        availability_zone=az,
        map_public_ip_on_launch=True,
    )
    subnet_ids.append(subnet.id)

# EKS Cluster
cluster = eks.Cluster(
    "eks-cluster",
    vpc_id=vpc.id,
    subnet_ids=subnet_ids,
    instance_type=node_type,
    desired_capacity=node_count,
    min_size=node_count,
    max_size=node_count * 2,
)

# Kubernetes provider
k8s_provider = k8s.Provider(
    "k8s-provider",
    kubeconfig=cluster.kubeconfig,
)

# Deploy app to Kubernetes
app = k8s.apps.v1.Deployment(
    "nginx-deployment",
    spec=k8s.apps.v1.DeploymentSpecArgs(
        replicas=3,
        selector=k8s.meta.v1.LabelSelectorArgs(
            match_labels={"app": "nginx"},
        ),
        template=k8s.core.v1.PodTemplateSpecArgs(
            metadata=k8s.meta.v1.ObjectMetaArgs(
                labels={"app": "nginx"},
            ),
            spec=k8s.core.v1.PodSpecArgs(
                containers=[
                    k8s.core.v1.ContainerArgs(
                        name="nginx",
                        image="nginx:latest",
                        ports=[k8s.core.v1.ContainerPortArgs(container_port=80)],
                    )
                ],
            ),
        ),
    ),
    opts=pulumi.ResourceOptions(provider=k8s_provider),
)

# Service
service = k8s.core.v1.Service(
    "nginx-service",
    spec=k8s.core.v1.ServiceSpecArgs(
        type="LoadBalancer",
        selector={"app": "nginx"},
        ports=[k8s.core.v1.ServicePortArgs(port=80, target_port=80)],
    ),
    opts=pulumi.ResourceOptions(provider=k8s_provider),
)

# Outputs
pulumi.export("kubeconfig", cluster.kubeconfig)
pulumi.export("cluster_name", cluster.eks_cluster.name)
pulumi.export("service_endpoint", service.status.load_balancer.ingress[0].hostname)

Pulumi Workflow

# Install Pulumi
$ curl -fsSL https://get.pulumi.com | sh

# Login (uses SaaS backend by default)
$ pulumi login
# Or use self-hosted: pulumi login s3://my-state-bucket

# Create new project
$ pulumi new aws-typescript
# or: aws-python, azure-python, kubernetes-go, etc.

# Install dependencies
$ npm install  # or: pip install -r requirements.txt

# Preview changes
$ pulumi preview

# Deploy
$ pulumi up

# View outputs
$ pulumi stack output

# Destroy
$ pulumi destroy

# Manage stacks (like environments)
$ pulumi stack init dev
$ pulumi stack select prod
$ pulumi stack ls

Unit Testing Infrastructure

# infrastructure.test.ts

import * as pulumi from "@pulumi/pulumi";
import { expect } from "chai";
import "mocha";

// Mock Pulumi runtime
pulumi.runtime.setMocks({
  newResource: (args: pulumi.runtime.MockResourceArgs) => {
    return {
      id: args.name + "_id",
      state: args.inputs,
    };
  },
  call: (args: pulumi.runtime.MockCallArgs) => {
    return args.inputs;
  },
});

describe("Infrastructure Tests", () => {
  let infra: typeof import("./index");

  before(async () => {
    infra = await import("./index");
  });

  it("should create VPC with correct CIDR", (done) => {
    pulumi.all([infra.vpcId]).apply(([vpcId]) => {
      expect(vpcId).to.not.be.undefined;
      done();
    });
  });

  it("should create correct number of instances", (done) => {
    pulumi.all([infra.instanceCount]).apply(([count]) => {
      expect(count).to.equal(2);
      done();
    });
  });
});

// Run: npm test

Pulumi Advantages:
• Multi-cloud (AWS, GCP, Azure, K8s, 100+ providers)
• Real programming languages with full ecosystem
• Unit testing with Jest, pytest, etc.
• Direct state management (no CloudFormation)
• Secrets management built-in
• Policy as code (Pulumi CrossGuard)

Considerations:
• Newer than Terraform (smaller community)
• SaaS backend (or self-host state)
• Learning curve for declarative → imperative thinking

AWS CloudFormation, AWS Native Declarative IaC

CloudFormation is AWS's native IaC service. It uses JSON or YAML templates to declare infrastructure. It's deeply integrated with AWS but verbose and limited compared to programmatic options.

CloudFormation Template

# template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Web application infrastructure'

Parameters:
  EnvironmentName:
    Type: String
    AllowedValues: [dev, staging, prod]
    Default: dev

  InstanceType:
    Type: String
    Default: t3.micro
    AllowedValues: [t3.micro, t3.small, t3.medium]

  KeyName:
    Type: AWS::EC2::KeyPair::KeyName
    Description: EC2 Key Pair

Mappings:
  EnvironmentConfig:
    dev:
      InstanceCount: 1
      DBInstanceClass: db.t4g.micro
    staging:
      InstanceCount: 2
      DBInstanceClass: db.t4g.small
    prod:
      InstanceCount: 3
      DBInstanceClass: db.t4g.medium

Conditions:
  IsProduction: !Equals [!Ref EnvironmentName, prod]
  CreateDatabase: !Not [!Equals [!Ref EnvironmentName, dev]]

Resources:
  # VPC
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: !Sub '${EnvironmentName}-vpc'

  # Public Subnet
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: 10.0.1.0/24
      AvailabilityZone: !Select [0, !GetAZs '']
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub '${EnvironmentName}-public-1'

  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: 10.0.2.0/24
      AvailabilityZone: !Select [1, !GetAZs '']
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub '${EnvironmentName}-public-2'

  # Internet Gateway
  InternetGateway:
    Type: AWS::EC2::InternetGateway

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

  # Security Group
  WebServerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for web servers
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0

  # Launch Template
  LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateName: !Sub '${EnvironmentName}-template'
      LaunchTemplateData:
        ImageId: !Sub '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2}}'
        InstanceType: !Ref InstanceType
        KeyName: !Ref KeyName
        SecurityGroupIds:
          - !Ref WebServerSecurityGroup
        UserData:
          Fn::Base64: !Sub |
            #!/bin/bash
            yum update -y
            yum install -y httpd
            systemctl start httpd
            systemctl enable httpd
            echo "<h1>Hello from ${EnvironmentName}</h1>" > /var/www/html/index.html

  # Auto Scaling Group
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplate
        Version: !GetAtt LaunchTemplate.LatestVersionNumber
      MinSize: !FindInMap [EnvironmentConfig, !Ref EnvironmentName, InstanceCount]
      MaxSize: !If [IsProduction, 10, 5]
      DesiredCapacity: !FindInMap [EnvironmentConfig, !Ref EnvironmentName, InstanceCount]
      VPCZoneIdentifier:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      TargetGroupARNs:
        - !Ref TargetGroup
      Tags:
        - Key: Name
          Value: !Sub '${EnvironmentName}-web'
          PropagateAtLaunch: true

  # Load Balancer
  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Subnets:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroups:
        - !Ref WebServerSecurityGroup

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      VpcId: !Ref VPC
      Port: 80
      Protocol: HTTP
      HealthCheckPath: /
      HealthCheckIntervalSeconds: 30

  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref LoadBalancer
      Port: 80
      Protocol: HTTP
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref TargetGroup

  # RDS Database (conditional)
  Database:
    Type: AWS::RDS::DBInstance
    Condition: CreateDatabase
    Properties:
      DBInstanceClass: !FindInMap [EnvironmentConfig, !Ref EnvironmentName, DBInstanceClass]
      Engine: postgres
      EngineVersion: '15.3'
      MasterUsername: admin
      MasterUserPassword: !Sub '{{resolve:secretsmanager:db-password}}'
      AllocatedStorage: !If [IsProduction, 100, 20]
      MultiAZ: !If [IsProduction, true, false]
      BackupRetentionPeriod: !If [IsProduction, 7, 1]
      VPCSecurityGroups:
        - !Ref WebServerSecurityGroup

Outputs:
  LoadBalancerDNS:
    Description: DNS name of the load balancer
    Value: !GetAtt LoadBalancer.DNSName
    Export:
      Name: !Sub '${AWS::StackName}-LoadBalancerDNS'

  VpcId:
    Description: VPC ID
    Value: !Ref VPC
    Export:
      Name: !Sub '${AWS::StackName}-VpcId'

CloudFormation CLI

# Validate template
$ aws cloudformation validate-template --template-body file://template.yaml

# Create stack
$ aws cloudformation create-stack \
  --stack-name my-app-prod \
  --template-body file://template.yaml \
  --parameters ParameterKey=EnvironmentName,ParameterValue=prod \
  --capabilities CAPABILITY_IAM

# Update stack
$ aws cloudformation update-stack \
  --stack-name my-app-prod \
  --template-body file://template.yaml \
  --parameters ParameterKey=EnvironmentName,ParameterValue=prod

# Delete stack
$ aws cloudformation delete-stack --stack-name my-app-prod

# Describe stack
$ aws cloudformation describe-stacks --stack-name my-app-prod

# List stacks
$ aws cloudformation list-stacks

# Get outputs
$ aws cloudformation describe-stacks \
  --stack-name my-app-prod \
  --query 'Stacks[0].Outputs'

CloudFormation Strengths:
• Native AWS integration
• No external dependencies
• Rollback on failure
• Change sets for preview

Limitations:
• Verbose YAML/JSON
• Limited reusability
• AWS-only
• Intrinsic functions are clunky
• Hard to test
→ Consider using AWS CDK instead (same backend, better DX)

Ansible, Configuration Management & Provisioning

Ansible is different from the above. It's for configuration management, installing packages, configuring services, managing files on existing servers. It's agentless (SSH-based) and uses YAML playbooks.

Ansible Playbook

# playbook.yml

---
- name: Configure web servers
  hosts: webservers
  become: yes
  vars:
    app_version: "1.2.3"
    nginx_port: 80

  tasks:
    # Update packages
    - name: Update apt cache
      apt:
        update_cache: yes
        cache_valid_time: 3600

    # Install packages
    - name: Install required packages
      apt:
        name:
          - nginx
          - python3-pip
          - git
          - ufw
        state: present

    # Configure firewall
    - name: Allow HTTP
      ufw:
        rule: allow
        port: '80'
        proto: tcp

    - name: Allow HTTPS
      ufw:
        rule: allow
        port: '443'
        proto: tcp

    - name: Enable UFW
      ufw:
        state: enabled

    # Deploy application
    - name: Clone application repository
      git:
        repo: 'https://github.com/myorg/myapp.git'
        dest: /var/www/myapp
        version: "{{ app_version }}"
      notify: Restart nginx

    # Install Python dependencies
    - name: Install app dependencies
      pip:
        requirements: /var/www/myapp/requirements.txt
        virtualenv: /var/www/myapp/venv

    # Configure Nginx
    - name: Copy nginx config
      template:
        src: templates/nginx.conf.j2
        dest: /etc/nginx/sites-available/myapp
      notify: Restart nginx

    - name: Enable site
      file:
        src: /etc/nginx/sites-available/myapp
        dest: /etc/nginx/sites-enabled/myapp
        state: link
      notify: Restart nginx

    # Systemd service
    - name: Copy systemd service file
      template:
        src: templates/myapp.service.j2
        dest: /etc/systemd/system/myapp.service
      notify: Restart app

    - name: Enable and start application
      systemd:
        name: myapp
        enabled: yes
        state: started
        daemon_reload: yes

  handlers:
    - name: Restart nginx
      systemd:
        name: nginx
        state: restarted

    - name: Restart app
      systemd:
        name: myapp
        state: restarted

# inventory.ini

[webservers]
web1.example.com ansible_host=192.168.1.10
web2.example.com ansible_host=192.168.1.11
web3.example.com ansible_host=192.168.1.12

[dbservers]
db1.example.com ansible_host=192.168.1.20

[production:children]
webservers
dbservers

[production:vars]
ansible_user=ubuntu
ansible_ssh_private_key_file=~/.ssh/id_rsa

Ansible Commands

# Test connectivity
$ ansible all -i inventory.ini -m ping

# Run playbook
$ ansible-playbook -i inventory.ini playbook.yml

# Run with specific tags
$ ansible-playbook -i inventory.ini playbook.yml --tags "deploy"

# Dry run (check mode)
$ ansible-playbook -i inventory.ini playbook.yml --check

# Limit to specific hosts
$ ansible-playbook -i inventory.ini playbook.yml --limit web1.example.com

# Run ad-hoc command
$ ansible webservers -i inventory.ini -m shell -a "uptime"

# Gather facts
$ ansible webservers -i inventory.ini -m setup

Ansible Roles (Reusability)

# Directory structure
roles/
  nginx/
    tasks/
      main.yml
    templates/
      nginx.conf.j2
    handlers/
      main.yml
    defaults/
      main.yml
    vars/
      main.yml

# roles/nginx/tasks/main.yml
---
- name: Install nginx
  apt:
    name: nginx
    state: present

- name: Copy config
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
  notify: restart nginx

# Use role in playbook
---
- name: Configure servers
  hosts: webservers
  roles:
    - nginx
    - postgresql
    - myapp

Ansible Use Cases:
• Configure servers (install packages, set up services)
• Application deployment
• Security hardening
• Patch management

When to use:
• Existing servers that need configuration
• Multi-cloud configuration management
• Simple deployments

Not for:
• Provisioning cloud infrastructure (use Terraform/CDK/Pulumi)
• Complex state management

IaC Best Practices

Lessons learned from managing infrastructure as code at scale.

1. Version Control Everything

# Git repository structure
infrastructure/
├── .gitignore          # Ignore state files, secrets
├── README.md           # Documentation
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   └── terraform.tfvars
│   ├── staging/
│   │   ├── main.tf
│   │   └── terraform.tfvars
│   └── prod/
│       ├── main.tf
│       └── terraform.tfvars
├── modules/
│   ├── vpc/
│   ├── ecs/
│   └── rds/
└── .github/
    └── workflows/
        └── terraform.yml

# .gitignore
*.tfstate
*.tfstate.backup
.terraform/
*.tfvars  # Don't commit secrets!

2. Remote State Management

# Terraform remote state (S3 + DynamoDB locking)

terraform {
  backend "s3" {
    bucket         = "mycompany-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"  # State locking
  }
}

# Why remote state?
# ✓ Team collaboration
# ✓ State locking (prevents concurrent modifications)
# ✓ Encryption at rest
# ✓ Backup and versioning
# ✓ Audit trail

3. Environment Separation

❌ Don't

Single codebase, conditionals everywhere

resource "aws_instance" "web" {
  count = var.env == "prod" ? 5 : 1
  instance_type = var.env == "prod" ?
    "t3.large" : "t3.micro"
}

Risky: Easy to break prod

✅ Do

Separate directories/stacks per environment

environments/
├── dev/
├── staging/
└── prod/

# Or separate repos/projects

Safe: Blast radius limited

4. Use Modules/Constructs

# DRY principle: Don't repeat yourself

# ❌ Copy-paste across environments
# environments/dev/main.tf
# environments/staging/main.tf
# environments/prod/main.tf
# (All with duplicate code)

# ✅ Create reusable modules
module "vpc" {
  source = "../../modules/vpc"
  environment = "prod"
  cidr_block = "10.0.0.0/16"
}

# Or use CDK constructs (even better)
const vpc = new Vpc(this, 'VPC', {
  maxAzs: 3
});

# Or Pulumi ComponentResource
class WebApp extends pulumi.ComponentResource { ... }

5. Automate with CI/CD

# .github/workflows/terraform.yml

name: Terraform

on:
  pull_request:
    paths: ['infrastructure/**']
  push:
    branches: [main]
    paths: ['infrastructure/**']

jobs:
  terraform:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infrastructure/environments/prod

    steps:
      - uses: actions/checkout@v4

      - uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        run: terraform init

      - name: Terraform Format Check
        run: terraform fmt -check

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan -out=plan.out
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply plan.out

6. Test Infrastructure Code

# CDK unit test example

import { Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

test('Creates VPC with 3 AZs', () => {
  const app = new cdk.App();
  const stack = new MyStack(app, 'TestStack', {
    environment: 'prod',
    instanceCount: 3,
  });

  const template = Template.fromStack(stack);

  // Assert VPC exists
  template.hasResourceProperties('AWS::EC2::VPC', {
    CidrBlock: '10.0.0.0/16',
  });

  // Assert 3 subnets
  template.resourceCountIs('AWS::EC2::Subnet', 3);
});

Golden Rules:
1. Never modify infrastructure manually (always through code)
2. Always use remote state with locking
3. Separate environments completely
4. Code review all infrastructure changes
5. Tag everything for cost allocation
6. Use least privilege IAM roles
7. Encrypt secrets (never commit to Git)
8. Document with comments and README
9. Plan before apply (preview changes)
10. Test in dev/staging before prod

Key Takeaways

IaC Philosophy: Infrastructure defined in code, versioned in Git, deployed automatically
Declarative (Terraform, CloudFormation): Configuration files describe desired state
Programmatic (AWS CDK, Pulumi): Real programming languages with full ecosystem
Terraform: Multi-cloud, HCL, huge community, but limited expressiveness
AWS CDK: TypeScript/Python for AWS, synthesizes to CloudFormation, powerful abstractions
Pulumi: Multi-cloud programmatic IaC, supports AWS/GCP/Azure/K8s, unit testable
CloudFormation: AWS native, verbose YAML/JSON, consider CDK instead
Ansible: Configuration management for existing servers, not infrastructure provisioning
Best Practices: Version control, remote state, environment separation, CI/CD, testing
Modern Choice: Use programmatic tools (CDK/Pulumi) for new projects, full programming power

When to Use Each Tool

Tool	Best For	Avoid When
AWS CDK	AWS-only projects, teams comfortable with TS/Python, need complex abstractions	Multi-cloud requirements, need CloudFormation-independent state
Pulumi	Multi-cloud, want programmatic IaC, team knows real programming	Team only knows declarative config, want largest community
Terraform	Multi-cloud, declarative mindset, want huge community/modules	Need complex logic, heavy testing, dynamic infrastructure
CloudFormation	AWS-only, no external tools allowed, simple use cases	Complex infrastructure, use CDK instead
Ansible	Configure existing servers, application deployment, multi-cloud config	Provisioning infrastructure (use Terraform/CDK/Pulumi)

Modern Recommendation

For new projects, strongly consider programmatic IaC (AWS CDK or Pulumi) over declarative tools. The ability to use real programming languages, with loops, functions, classes, type safety, and full IDE support, makes infrastructure code more maintainable, testable, and expressive. Terraform is industry standard and battle-tested, but its DSL limitations become painful as infrastructure grows complex.

Back to "CI/CD Pipelines"

Infrastructure for EngineersLesson 8

Next: "Cloud Platforms (AWS/GCP/Azure)"