Kubernetes Fundamentals

From pods to production clusters, master container orchestration at scale

Why Kubernetes?

Docker lets you run containers. Kubernetes (K8s) lets you run containers at scale, managing hundreds or thousands of containers across multiple machines, handling failures automatically, scaling workloads based on demand, and providing zero-downtime deployments. If Docker is your engine, Kubernetes is your entire transportation system: routing, load balancing, self-healing, and orchestration.

Kubernetes Architecture, The Big Picture

Kubernetes is a cluster of machines (physical or virtual) working together to run containers. The cluster has two types of nodes: control plane (the brain) and worker nodes (the muscle).

KUBERNETES CLUSTER ARCHITECTURE

┌─────────────────────────────────────────────────────────────────┐
│                    CONTROL PLANE (Master)                       │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────────┐    │
│  │ API Server   │  │  Scheduler   │  │  Controller Manager │    │
│  │ (Frontend)   │  │ (Placement)  │  │  (State reconciler) │    │
│  └──────────────┘  └──────────────┘  └─────────────────────┘    │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │              etcd (Cluster State Database)               │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ API calls
                              │
        ┌─────────────────────┼────────────────────┐
        │                     │                    │
┌───────▼────────┐   ┌────────▼───────┐   ┌────────▼───────┐
│  WORKER NODE 1 │   │ WORKER NODE 2  │   │ WORKER NODE 3  │
│  ┌──────────┐  │   │  ┌──────────┐  │   │  ┌──────────┐  │
│  │ kubelet  │  │   │  │ kubelet  │  │   │  │ kubelet  │  │
│  │ (agent)  │  │   │  │ (agent)  │  │   │  │ (agent)  │  │
│  └──────────┘  │   │  └──────────┘  │   │  └──────────┘  │
│  ┌──────────┐  │   │  ┌──────────┐  │   │  ┌──────────┐  │
│  │kube-proxy│  │   │  │kube-proxy│  │   │  │kube-proxy│  │
│  └──────────┘  │   │  └──────────┘  │   │  └──────────┘  │
│                │   │                │   │                │
│  [Container]   │   │  [Container]   │   │  [Container]   │
│  [Container]   │   │  [Container]   │   │  [Container]   │
│  [Container]   │   │  [Container]   │   │  [Container]   │
└────────────────┘   └────────────────┘   └────────────────┘

Think of it like a company:
• Control Plane = Management (decides what work goes where)
• Worker Nodes = Employees (actually do the work)
• kubelet = Worker who executes tasks
• API Server = HR department (all requests go through here)

Control Plane Components

Component	What It Does
API Server	Front-end for Kubernetes. All kubectl commands go here. Validates and processes requests.
etcd	Distributed key-value store. Holds ALL cluster state (what pods exist, where they run, etc.)
Scheduler	Watches for new pods with no assigned node. Selects best node based on resources, constraints.
Controller Manager	Runs controllers that watch cluster state and make changes (e.g., ensures 3 replicas = 3 running pods)

Worker Node Components

Component	What It Does
kubelet	Agent running on each node. Receives pod specs from API server and ensures containers are running.
kube-proxy	Network proxy. Maintains network rules for pod communication and load balancing.
Container Runtime	Software that runs containers (Docker, containerd, CRI-O). kubelet talks to this.

Pods, The Smallest Deployable Unit

A Pod is the smallest unit in Kubernetes. It wraps one or more containers that share network and storage. Think of it as a "container of containers."

Pod Characteristics

Containers in a pod share network (same IP)
Share storage volumes
Always scheduled together on same node
Ephemeral by nature (pods die, don't heal)
Each pod gets unique IP address

Common Patterns

One container per pod: Most common (90%)
Sidecar pattern: Main + helper (logging, proxy)
Adapter pattern: Main + data transformer
Ambassador pattern: Main + proxy to external

Creating Your First Pod

# nginx-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.25-alpine
    ports:
    - containerPort: 80

# Create the pod
$ kubectl apply -f nginx-pod.yaml
pod/nginx-pod created

# View pods
$ kubectl get pods
NAME        READY   STATUS    RESTARTS   AGE
nginx-pod   1/1     Running   0          10s

# Get detailed info
$ kubectl describe pod nginx-pod

# View logs
$ kubectl logs nginx-pod

# Execute command in pod
$ kubectl exec -it nginx-pod -- sh
/ # ls
/ # exit

# Delete pod
$ kubectl delete pod nginx-pod

What just happened:
1. Defined pod spec in YAML
2. API server validated it
3. Scheduler placed it on a node
4. kubelet on that node pulled image and started container

Multi-Container Pod Example (Sidecar Pattern)

# app-with-sidecar.yaml

apiVersion: v1
kind: Pod
metadata:
  name: app-with-logging
spec:
  containers:
  # Main application container
  - name: app
    image: my-app:1.0
    ports:
    - containerPort: 8080
    volumeMounts:
    - name: logs
      mountPath: /var/log/app
  
  # Sidecar: log shipper
  - name: log-shipper
    image: fluent/fluent-bit:2.0
    volumeMounts:
    - name: logs
      mountPath: /var/log/app
      readOnly: true
  
  volumes:
  - name: logs
    emptyDir: {}

Use case: App writes logs to /var/log/app. Log shipper (sidecar) reads those logs and sends them to Elasticsearch/CloudWatch. Both containers share the same volume.

Deployments, Managing Pod Replicas

You almost never create pods directly. Instead, you use Deployments which create ReplicaSets which create Pods. This hierarchy enables rolling updates, rollbacks, and scaling.

DEPLOYMENT (you manage this)
    │
    ├── ReplicaSet v2 (current)
    │   ├── Pod 1
    │   ├── Pod 2
    │   └── Pod 3
    │
    └── ReplicaSet v1 (old, kept for rollback)
        └── (scaled to 0)

Flow:
1. You create/update Deployment
2. Deployment creates/updates ReplicaSet
3. ReplicaSet creates/manages Pods

Creating a Deployment

# nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3  # Keep 3 pods running
  selector:
    matchLabels:
      app: nginx
  template:  # Pod template
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.25-alpine
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "128Mi"
            cpu: "500m"

# Create deployment
$ kubectl apply -f nginx-deployment.yaml
deployment.apps/nginx-deployment created

# Watch it create pods
$ kubectl get deployments
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3/3     3            3           30s

$ kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
nginx-deployment-7d4c8f9b6-4xj2k   1/1     Running   0          35s
nginx-deployment-7d4c8f9b6-mx8p7   1/1     Running   0          35s
nginx-deployment-7d4c8f9b6-zt9qw   1/1     Running   0          35s

# View ReplicaSet (created automatically)
$ kubectl get replicasets
NAME                         DESIRED   CURRENT   READY   AGE
nginx-deployment-7d4c8f9b6   3         3         3       40s

Self-healing in action: Delete a pod manually. Deployment will immediately create a new one to maintain the desired count of 3.

Scaling Deployments

# Scale up to 5 replicas
$ kubectl scale deployment nginx-deployment --replicas=5
deployment.apps/nginx-deployment scaled

$ kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
nginx-deployment-7d4c8f9b6-4xj2k   1/1     Running   0          5m
nginx-deployment-7d4c8f9b6-mx8p7   1/1     Running   0          5m
nginx-deployment-7d4c8f9b6-zt9qw   1/1     Running   0          5m
nginx-deployment-7d4c8f9b6-7ph5n   1/1     Running   0          5s
nginx-deployment-7d4c8f9b6-k2m9x   1/1     Running   0          5s

# Scale down to 2
$ kubectl scale deployment nginx-deployment --replicas=2

# Or edit the YAML and re-apply
$ kubectl edit deployment nginx-deployment

Rolling Updates, Zero-Downtime Deployments

# Update image version
$ kubectl set image deployment/nginx-deployment nginx=nginx:1.26-alpine
deployment.apps/nginx-deployment image updated

# Watch the rollout
$ kubectl rollout status deployment/nginx-deployment
Waiting for deployment "nginx-deployment" rollout to finish: 1 out of 3 new replicas...
Waiting for deployment "nginx-deployment" rollout to finish: 2 out of 3 new replicas...
deployment "nginx-deployment" successfully rolled out

# View rollout history
$ kubectl rollout history deployment/nginx-deployment
REVISION  CHANGE-CAUSE
1         <none>
2         <none>

# Rollback to previous version
$ kubectl rollout undo deployment/nginx-deployment
deployment.apps/nginx-deployment rolled back

# Rollback to specific revision
$ kubectl rollout undo deployment/nginx-deployment --to-revision=1

How rolling updates work:
1. Create new ReplicaSet with new version
2. Gradually scale up new, scale down old (default: 25% at a time)
3. Wait for health checks before continuing
4. Keep old ReplicaSet for easy rollback

Services, Stable Network Access to Pods

Pods are ephemeral and have changing IPs. Services provide a stable endpoint (DNS name + IP) for accessing a group of pods, with built-in load balancing.

Service Types

Type	What It Does	Use Case
ClusterIP	Internal IP only, accessible within cluster	Default. Backend services, databases
NodePort	Exposes service on each node's IP at a static port	Development, testing, simple external access
LoadBalancer	Cloud provider creates external load balancer	Production external access (AWS ELB, GCP LB)
ExternalName	Maps service to DNS name	Proxy to external service outside cluster

ClusterIP Service (Internal)

# backend-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  type: ClusterIP  # Default, can be omitted
  selector:
    app: backend  # Targets pods with this label
  ports:
  - port: 80          # Service port
    targetPort: 8080  # Pod port

$ kubectl apply -f backend-service.yaml
service/backend-service created

$ kubectl get services
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
backend-service   ClusterIP   10.96.143.201   <none>        80/TCP    10s

# Other pods can now access via:
# - DNS: http://backend-service
# - DNS: http://backend-service.default.svc.cluster.local
# - IP:  http://10.96.143.201

Load balancing: Service automatically load-balances across all pods matching the selector. As pods come and go, service endpoints update automatically.

NodePort Service (External - Dev/Test)

# frontend-nodeport.yaml

apiVersion: v1
kind: Service
metadata:
  name: frontend-service
spec:
  type: NodePort
  selector:
    app: frontend
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080  # Optional: specify port 30000-32767

$ kubectl apply -f frontend-nodeport.yaml
service/frontend-service created

$ kubectl get services
NAME               TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
frontend-service   NodePort   10.96.45.123   <none>        80:30080/TCP   5s

# Access from outside cluster:
# http://<any-node-ip>:30080

LoadBalancer Service (Production)

# web-loadbalancer.yaml

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 8080

$ kubectl apply -f web-loadbalancer.yaml
service/web-service created

$ kubectl get services
NAME          TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
web-service   LoadBalancer   10.96.78.234   34.120.145.67    80:31245/TCP   2m

# Cloud provider created load balancer
# Access via: http://34.120.145.67

How it works: Kubernetes talks to cloud provider API (AWS/GCP/Azure) to provision an actual load balancer. Traffic → LB → NodePort → Service → Pods

ConfigMaps & Secrets, Configuration Management

Never hardcode configuration in container images. Use ConfigMaps for non-sensitive config and Secrets for sensitive data.

ConfigMaps

# app-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  APP_ENV: "production"
  LOG_LEVEL: "info"
  MAX_CONNECTIONS: "100"
  config.json: |
    {
      "feature_flags": {
        "new_ui": true,
        "beta_api": false
      }
    }

# Using ConfigMap in Pod

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: my-app:1.0
    
    # Option 1: Inject as environment variables
    envFrom:
    - configMapRef:
        name: app-config
    
    # Option 2: Mount as files
    volumeMounts:
    - name: config
      mountPath: /etc/config
  
  volumes:
  - name: config
    configMap:
      name: app-config

Secrets (Sensitive Data)

# Create secret from literal values
$ kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=superSecret123

# Create from file
$ kubectl create secret generic api-key \
  --from-file=key.txt

# View secrets (values are base64 encoded)
$ kubectl get secrets
NAME              TYPE     DATA   AGE
db-credentials    Opaque   2      30s

# Using Secrets in Pod

apiVersion: v1
kind: Pod
metadata:
  name: db-client
spec:
  containers:
  - name: app
    image: my-app:1.0
    env:
    - name: DB_USERNAME
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: username
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: password

Security Note

Kubernetes Secrets are base64 encoded, NOT encrypted by default. For production, enable encryption at rest or use external secret managers (AWS Secrets Manager, HashiCorp Vault, Sealed Secrets).

Namespaces, Cluster Isolation

Namespaces provide virtual clusters within a physical cluster. They isolate resources, enable multi-tenancy, and help organize large deployments.

# List namespaces
$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   30d
kube-system       Active   30d  # K8s system components
kube-public       Active   30d  # Publicly readable
kube-node-lease   Active   30d  # Node heartbeats

# Create namespace
$ kubectl create namespace development
$ kubectl create namespace production

# Or via YAML
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: staging
EOF

# Deploy to specific namespace
$ kubectl apply -f deployment.yaml -n development

# Set default namespace for current context
$ kubectl config set-context --current --namespace=development

# View resources in namespace
$ kubectl get pods -n production
$ kubectl get all -n staging

Common namespace strategy:
• dev, staging, prod (environment separation)
• team-a, team-b (team isolation)
• app-frontend, app-backend (application components)

Resource Quotas (Namespace Limits)

# dev-quota.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: dev-quota
  namespace: development
spec:
  hard:
    pods: "10"              # Max 10 pods
    requests.cpu: "4"       # Max 4 CPU cores requested
    requests.memory: "8Gi"  # Max 8GB memory requested
    limits.cpu: "8"         # Max 8 CPU cores limit
    limits.memory: "16Gi"   # Max 16GB memory limit

Why use quotas? Prevent teams from consuming all cluster resources. Essential for multi-tenant clusters.

Health Checks, Probes for Reliability

Kubernetes needs to know if your app is healthy. Probes tell Kubernetes when to restart containers or when they're ready to receive traffic.

Probe Type	Purpose	When It Fails
Liveness Probe	Is the container alive?	Kubernetes restarts the container
Readiness Probe	Is the container ready for traffic?	Removed from service endpoints (no traffic)
Startup Probe	Has the container started successfully?	Other probes don't run until this passes

# deployment-with-probes.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: my-api:1.0
        ports:
        - containerPort: 8080
        
        # Liveness: Check if app is alive
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 30  # Wait 30s before first check
          periodSeconds: 10         # Check every 10s
          timeoutSeconds: 5         # Timeout after 5s
          failureThreshold: 3       # Restart after 3 failures
        
        # Readiness: Check if ready for traffic
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
          failureThreshold: 3
        
        # Startup: For slow-starting apps
        startupProbe:
          httpGet:
            path: /startup
            port: 8080
          failureThreshold: 30      # 30 attempts
          periodSeconds: 10          # = 5 minutes max startup time

Probe Mechanisms

HTTP GET

Sends HTTP request to specified path/port

httpGet:
&nbsp;&nbsp;path: /health
&nbsp;&nbsp;port: 8080

TCP Socket

Checks if port is open

tcpSocket:
&nbsp;&nbsp;port: 5432

Exec Command

Runs command in container

exec:
&nbsp;&nbsp;command:
&nbsp;&nbsp;- cat
&nbsp;&nbsp;- /tmp/healthy

Volumes, Persistent Storage

Pods are ephemeral, but data often isn't. Kubernetes provides various volume typesto persist data beyond pod lifetime.

Volume Type	Use Case
emptyDir	Temporary storage, pod lifetime only. Shared between containers in pod.
hostPath	Mount host directory into pod. Dangerous, avoid in production.
PersistentVolume	Cluster-level storage resource. Survives pod deletion.
Cloud Volumes	AWS EBS, GCP Persistent Disk, Azure Disk, etc.

PersistentVolume & PersistentVolumeClaim

Storage Workflow:
1. Admin creates PersistentVolume (PV) - actual storage
2. User creates PersistentVolumeClaim (PVC) - request for storage
3. Kubernetes binds PVC to matching PV
4. Pod references PVC in its spec

   ┌──────────────────┐
   │ PersistentVolume │  ← Admin provisions (100GB SSD)
   └────────┬─────────┘
            │ Binding
   ┌────────▼─────────┐
   │      PVC         │  ← User requests (10GB)
   └────────┬─────────┘
            │
   ┌────────▼─────────┐
   │      Pod         │  ← Uses PVC
   └──────────────────┘

# pv-example.yaml (Admin creates)

apiVersion: v1
kind: PersistentVolume
metadata:
  name: postgres-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
  - ReadWriteOnce  # Single node read-write
  persistentVolumeReclaimPolicy: Retain
  storageClassName: fast-ssd
  hostPath:
    path: /mnt/data  # For demo only, use cloud volumes in prod

# pvc-example.yaml (User creates)

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: fast-ssd

# Using PVC in Pod

apiVersion: v1
kind: Pod
metadata:
  name: postgres
spec:
  containers:
  - name: postgres
    image: postgres:16
    volumeMounts:
    - name: postgres-storage
      mountPath: /var/lib/postgresql/data
  volumes:
  - name: postgres-storage
    persistentVolumeClaim:
      claimName: postgres-pvc  # References PVC

Dynamic Provisioning: In cloud environments, you usually don't create PVs manually. Instead, StorageClasses automatically provision volumes when PVCs are created.

kubectl Essentials, Your Command Line Tool

kubectl is your primary interface to Kubernetes. Master these commands for daily operations.

Most Used kubectl Commands

# Create/Update resources
$ kubectl apply -f deployment.yaml
$ kubectl apply -f .  # All YAML files in directory

# View resources
$ kubectl get pods
$ kubectl get pods -o wide  # More details (node, IP)
$ kubectl get all           # All resources
$ kubectl get all -n production  # In specific namespace

# Describe (detailed info)
$ kubectl describe pod nginx-pod
$ kubectl describe deployment my-app

# View logs
$ kubectl logs pod-name
$ kubectl logs pod-name -f  # Follow (tail)
$ kubectl logs pod-name -c container-name  # Multi-container pod
$ kubectl logs deployment/my-app  # All pods in deployment

# Execute commands
$ kubectl exec -it pod-name -- bash
$ kubectl exec pod-name -- ls /app

# Port forwarding (local dev)
$ kubectl port-forward pod-name 8080:80
$ kubectl port-forward service/my-service 8080:80

# Delete resources
$ kubectl delete pod nginx-pod
$ kubectl delete -f deployment.yaml
$ kubectl delete deployment my-app

# Editing resources
$ kubectl edit deployment my-app  # Opens in editor
$ kubectl scale deployment my-app --replicas=5
$ kubectl set image deployment/my-app app=my-app:v2

# Debugging
$ kubectl get events  # Cluster events
$ kubectl top nodes   # Resource usage by nodes
$ kubectl top pods    # Resource usage by pods

kubectl Shortcuts & Aliases

# Add to ~/.bashrc or ~/.zshrc

alias k='kubectl'
alias kg='kubectl get'
alias kgp='kubectl get pods'
alias kd='kubectl describe'
alias kdp='kubectl describe pod'
alias kl='kubectl logs'
alias kx='kubectl exec -it'
alias ka='kubectl apply -f'

# Now you can:
$ k get pods
$ kl my-pod
$ kx my-pod -- bash

Context & Namespace Management

# View contexts (clusters)
$ kubectl config get-contexts

# Switch context
$ kubectl config use-context production-cluster

# Set default namespace
$ kubectl config set-context --current --namespace=development

# View current config
$ kubectl config view

# Quick namespace override
$ kubectl get pods -n kube-system

Pro-Tip: Install kubectx and kubens for easier context and namespace switching:
kubectx production (switch cluster)
kubens development (switch namespace)

10. Autoscaling, Dynamic Resource Adjustment

Kubernetes can automatically scale your applications based on metrics like CPU, memory, or custom metrics.

Horizontal Pod Autoscaler (HPA)

Automatically adjusts the number of pod replicas based on observed metrics.

# hpa.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70  # Scale when avg CPU > 70%

# Create HPA via command
$ kubectl autoscale deployment api-deployment \
  --min=2 --max=10 --cpu-percent=70

# View HPA status
$ kubectl get hpa
NAME      REFERENCE                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
api-hpa   Deployment/api-deployment 45%/70%   2         10        3          5m

# Describe HPA
$ kubectl describe hpa api-hpa

How it works:
1. Metrics server collects resource usage every 15 seconds
2. HPA controller checks metrics every 15 seconds
3. If CPU >70%, scale up. If CPU < 70%, scale down
4. Cooldown periods prevent flapping (scale up: 3min, scale down: 5min)

Cluster Autoscaler (CA)

Automatically adds or removes worker nodes when pods can't be scheduled or nodes are underutilized.

Scenario: HPA scales your deployment to 20 replicas
Problem: Only 15 pods can fit on existing nodes
Solution: Cluster Autoscaler adds 2 more nodes

Flow:
1. HPA scales deployment to 20 replicas
2. 5 pods are "Pending" (can't be scheduled)
3. CA detects pending pods
4. CA talks to cloud provider API
5. New nodes are provisioned
6. Pending pods are scheduled on new nodes

Scale Down:
- If nodes are <50% utilized for 10+ minutes
- CA marks node for deletion
- Gracefully drains pods to other nodes
- Deletes node

Cloud-specific: Cluster Autoscaler works with AWS Auto Scaling Groups, GCP Managed Instance Groups, Azure VMSS. Setup varies by provider.

11. Troubleshooting, Common Issues

When things break (and they will), here's how to diagnose and fix common Kubernetes issues.

Pod Status Troubleshooting

Status: ImagePullBackOff

Problem: Can't pull container image

$ kubectl describe pod pod-name
# Look for: "Failed to pull image" in events

Possible causes:
• Image name typo
• Image doesn't exist
• Registry authentication failed
• Private registry, need imagePullSecrets

Status: CrashLoopBackOff

Problem: Container starts then crashes repeatedly

$ kubectl logs pod-name
$ kubectl logs pod-name --previous  # Last crashed instance

Common causes:
• Application error (check logs)
• Missing environment variables
• Failed health checks
• Wrong command/entrypoint

Status: Pending

Problem: Pod can't be scheduled to a node

$ kubectl describe pod pod-name
# Look at "Events" section

Common reasons:
• Insufficient resources (CPU/memory)
• No nodes match node selector
• PersistentVolumeClaim not bound
• Pod affinity/anti-affinity rules

Can't Access Service

Problem: Service exists but can't reach pods

# Check service endpoints
$ kubectl get endpoints service-name

# Should list pod IPs. If empty:
• Service selector doesn't match pod labels
• Pods not ready (failing readiness probe)
• No pods running

# Verify labels match
$ kubectl get pods --show-labels
$ kubectl describe service service-name

Standard Debugging Workflow

# 1. Check pod status
$ kubectl get pods

# 2. Get detailed info
$ kubectl describe pod <pod-name>
# Look at: Events, State, Conditions

# 3. Check logs
$ kubectl logs <pod-name>
$ kubectl logs <pod-name> --previous  # If crashed

# 4. Check events
$ kubectl get events --sort-by='.lastTimestamp'

# 5. Exec into container (if running)
$ kubectl exec -it <pod-name> -- sh
# Test connectivity, check files, env vars

# 6. Check service endpoints
$ kubectl get endpoints <service-name>

# 7. Port forward for testing
$ kubectl port-forward pod/<pod-name> 8080:80

# 8. Check resource usage
$ kubectl top pods
$ kubectl top nodes

Pro-Tip: Most issues show up in kubectl describe events. Always check there first!

12. Best Practices & Production Tips

Production Kubernetes requires discipline. Follow these practices for reliable, maintainable clusters.

DO

Use Deployments, not bare Pods
Set resource requests and limits
Define health checks (liveness, readiness)
Use namespaces for isolation
Tag images with versions, not :latest
Store configs in ConfigMaps/Secrets
Use labels and annotations extensively
Implement monitoring and logging
Use RBAC for access control

DON'T

Run as root in containers
Use :latest tag in production
Skip resource limits (causes node crashes)
Hardcode values in YAML
Use hostPath in production
Deploy without health checks
Store secrets in environment variables
Ignore events and logs
Over-provision resources wastefully

Resource Requests & Limits Strategy

resources:
  requests:      # Minimum guaranteed
    memory: "256Mi"
    cpu: "250m"  # 0.25 CPU cores
  limits:        # Maximum allowed
    memory: "512Mi"
    cpu: "500m"

# Strategy:
# - requests: What you need under normal load
# - limits: Burst capacity (2x requests is common)
# - Memory limit = Hard limit (OOMKilled if exceeded)
# - CPU limit = Throttling (slows down, doesn't kill)

Why this matters: Requests determine scheduling. Limits prevent resource hogging. Without limits, one pod can consume all node resources and crash other pods.

Key Takeaways

Architecture: Control plane (brain) + worker nodes (muscle)
Pods: Smallest deployable unit, usually 1 container per pod
Deployments: Manage ReplicaSets which manage Pods (self-healing, scaling, rolling updates)
Services: Stable network endpoints with load balancing (ClusterIP, NodePort, LoadBalancer)
ConfigMaps & Secrets: Externalize configuration and sensitive data
Namespaces: Virtual clusters for isolation and multi-tenancy
Health Checks: Liveness, readiness, startup probes ensure reliability
Volumes: PersistentVolumes and PVCs for stateful workloads
Autoscaling: HPA scales pods, CA scales nodes automatically
kubectl: Your command-line Swiss Army knife for cluster management

Quick Reference

# Resource Management
kubectl apply -f deployment.yaml
kubectl get all
kubectl get pods -o wide
kubectl describe pod <name>
kubectl logs <pod> -f
kubectl exec -it <pod> -- bash
kubectl delete -f deployment.yaml

# Scaling
kubectl scale deployment <name> --replicas=5
kubectl autoscale deployment <name> --min=2 --max=10 --cpu-percent=70

# Updates & Rollbacks
kubectl set image deployment/<name> app=app:v2
kubectl rollout status deployment/<name>
kubectl rollout undo deployment/<name>
kubectl rollout history deployment/<name>

# Services
kubectl expose deployment <name> --port=80 --target-port=8080
kubectl port-forward service/<name> 8080:80

# Config & Secrets
kubectl create configmap <name> --from-literal=key=value
kubectl create secret generic <name> --from-literal=password=secret

# Debugging
kubectl get events --sort-by='.lastTimestamp'
kubectl top nodes
kubectl top pods
kubectl cluster-info

Back to "Docker & Containerization"

Infrastructure for EngineersLesson 5

Next: "Advanced Kubernetes"