Kubernetes Autoscaling Complete Guide (Part 2): Cluster Autoscaling & Cloud Providers

November 9, 2025 • 32 min read • Yen

Kubernetes K8S Autoscaling Cluster Autoscaler Karpenter EKS GKE Cloud Native Infrastructure Cost Optimization DevOps Cloud

Series Overview

This is Part 2 of the Kubernetes Autoscaling Complete Guide series:

Part 1: Horizontal Pod Autoscaler - Application-level autoscaling with HPA, custom metrics, and KEDA
Part 2 (This Post): Cluster Autoscaling & Cloud Providers - Infrastructure-level autoscaling with Cluster Autoscaler, Karpenter, and cloud-specific solutions

While Horizontal Pod Autoscaler (HPA) manages application-level scaling by adjusting pod replicas (covered in Part 1 ), production Kubernetes environments require intelligent cluster-level autoscaling that dynamically provisions and deprovisions compute resources. This comprehensive guide explores advanced autoscaling strategies across node management, cloud provider integrations, and cutting-edge autoscaling technologies.

The Complete Autoscaling Picture

Multi-Layer Autoscaling Architecture

Effective Kubernetes autoscaling operates across three interconnected layers:

┌─────────────────────────────────────────────────────────────────────────┐
│                   KUBERNETES AUTOSCALING LAYERS                        │
│                                                                         │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │  LAYER 3: APPLICATION AUTOSCALING                                │  │
│  │  • HPA (Horizontal Pod Autoscaler)                               │  │
│  │  • VPA (Vertical Pod Autoscaler)                                 │  │
│  │  • KEDA (Event-Driven Autoscaling)                              │  │
│  │  ↓ Scales pod replicas based on metrics                          │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│                              ↓                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │  LAYER 2: CLUSTER AUTOSCALING (This Guide's Focus)              │  │
│  │  • Cluster Autoscaler                                             │  │
│  │  • Karpenter                                                      │  │
│  │  • Cloud Provider Native Autoscaling                             │  │
│  │  ↓ Provisions/deprovisions nodes based on pod scheduling         │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│                              ↓                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │  LAYER 1: INFRASTRUCTURE AUTOSCALING                             │  │
│  │  • VM Instance Groups                                             │  │
│  │  • AWS Auto Scaling Groups                                        │  │
│  │  • Azure VM Scale Sets                                            │  │
│  │  ↓ Manages underlying compute infrastructure                      │  │
│  └──────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘

Why Cluster Autoscaling Matters

Business Impact:

Metric	Without Cluster Autoscaling	With Cluster Autoscaling
Infrastructure Costs	Over-provisioned 24/7	40-60% cost reduction
Incident Response	Manual node provisioning	Automated capacity addition
Resource Utilization	20-30% average utilization	60-80% utilization
Scaling Time	Hours (manual)	Minutes (automated)
Operational Burden	High (capacity planning)	Low (self-managing)

Approach 1: Kubernetes Cluster Autoscaler (CA)

Overview and Architecture

The Cluster Autoscaler is the official Kubernetes project that automatically adjusts cluster size based on pod scheduling needs. It’s the most mature and widely adopted cluster autoscaling solution.

How Cluster Autoscaler Works:

┌─────────────────────────────────────────────────────────────────────┐
│              CLUSTER AUTOSCALER DECISION FLOW                      │
│                                                                     │
│  Pod Created → Pending State → CA Detects → Check Node Groups      │
│       ↓             ↓              ↓               ↓                │
│  Scheduler     No Resources   Evaluation     Available Types       │
│  Attempts      Available      Logic          & Constraints         │
│       ↓             ↓              ↓               ↓                │
│  Fails to      Triggers CA    Simulates      Selects Best          │
│  Schedule      Scale-Up       Placement      Node Group            │
│       ↓             ↓              ↓               ↓                │
│  Remains       Provisions      Tests Fit      Expands Group        │
│  Pending       New Node       Scenarios       (Cloud API)          │
│                     ↓              ↓               ↓                │
│              Node Joins    Pod Scheduled    Pod Running            │
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │            SCALE-DOWN LOGIC (Proactive)                     │   │
│  │                                                              │   │
│  │  Every 10s: Check node utilization                          │   │
│  │    ↓                                                         │   │
│  │  Node < 50% utilized for 10+ minutes?                       │   │
│  │    ↓                                                         │   │
│  │  Can all pods be rescheduled elsewhere?                     │   │
│  │    ↓                                                         │   │
│  │  Safe to drain? (PDBs, local storage, etc.)                 │   │
│  │    ↓                                                         │   │
│  │  Cordon → Drain → Terminate Node                            │   │
│  └─────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Implementation: Cluster Autoscaler on Self-Managed Kubernetes

Step 1: IAM Setup (AWS Example)

 1{
 2  "Version": "2012-10-17",
 3  "Statement": [
 4    {
 5      "Effect": "Allow",
 6      "Action": [
 7        "autoscaling:DescribeAutoScalingGroups",
 8        "autoscaling:DescribeAutoScalingInstances",
 9        "autoscaling:DescribeLaunchConfigurations",
10        "autoscaling:DescribeScalingActivities",
11        "autoscaling:DescribeTags",
12        "ec2:DescribeInstanceTypes",
13        "ec2:DescribeLaunchTemplateVersions"
14      ],
15      "Resource": ["*"]
16    },
17    {
18      "Effect": "Allow",
19      "Action": [
20        "autoscaling:SetDesiredCapacity",
21        "autoscaling:TerminateInstanceInAutoScalingGroup",
22        "ec2:DescribeImages",
23        "ec2:GetInstanceTypesFromInstanceRequirements",
24        "eks:DescribeNodegroup"
25      ],
26      "Resource": ["*"]
27    }
28  ]
29}

Step 2: Auto Scaling Group Tags

 1# Tag ASG for Cluster Autoscaler discovery
 2aws autoscaling create-or-update-tags \
 3  --tags \
 4    ResourceId=my-asg-name \
 5    ResourceType=auto-scaling-group \
 6    Key=k8s.io/cluster-autoscaler/enabled \
 7    Value=true \
 8    PropagateAtLaunch=false \
 9  --tags \
10    ResourceId=my-asg-name \
11    ResourceType=auto-scaling-group \
12    Key=k8s.io/cluster-autoscaler/my-cluster-name \
13    Value=owned \
14    PropagateAtLaunch=false

Step 3: Cluster Autoscaler Deployment

  1apiVersion: v1
  2kind: ServiceAccount
  3metadata:
  4  name: cluster-autoscaler
  5  namespace: kube-system
  6  annotations:
  7    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/cluster-autoscaler-role
  8
  9---
 10apiVersion: rbac.authorization.k8s.io/v1
 11kind: ClusterRole
 12metadata:
 13  name: cluster-autoscaler
 14rules:
 15- apiGroups: [""]
 16  resources: ["events", "endpoints"]
 17  verbs: ["create", "patch"]
 18- apiGroups: [""]
 19  resources: ["pods/eviction"]
 20  verbs: ["create"]
 21- apiGroups: [""]
 22  resources: ["pods/status"]
 23  verbs: ["update"]
 24- apiGroups: [""]
 25  resources: ["endpoints"]
 26  resourceNames: ["cluster-autoscaler"]
 27  verbs: ["get", "update"]
 28- apiGroups: [""]
 29  resources: ["nodes"]
 30  verbs: ["watch", "list", "get", "update"]
 31- apiGroups: [""]
 32  resources: ["namespaces", "pods", "services", "replicationcontrollers", "persistentvolumeclaims", "persistentvolumes"]
 33  verbs: ["watch", "list", "get"]
 34- apiGroups: ["extensions"]
 35  resources: ["replicasets", "daemonsets"]
 36  verbs: ["watch", "list", "get"]
 37- apiGroups: ["policy"]
 38  resources: ["poddisruptionbudgets"]
 39  verbs: ["watch", "list"]
 40- apiGroups: ["apps"]
 41  resources: ["statefulsets", "replicasets", "daemonsets"]
 42  verbs: ["watch", "list", "get"]
 43- apiGroups: ["storage.k8s.io"]
 44  resources: ["storageclasses", "csinodes", "csidrivers", "csistoragecapacities"]
 45  verbs: ["watch", "list", "get"]
 46- apiGroups: ["batch"]
 47  resources: ["jobs", "cronjobs"]
 48  verbs: ["watch", "list", "get"]
 49- apiGroups: ["coordination.k8s.io"]
 50  resources: ["leases"]
 51  verbs: ["create"]
 52- apiGroups: ["coordination.k8s.io"]
 53  resourceNames: ["cluster-autoscaler"]
 54  resources: ["leases"]
 55  verbs: ["get", "update"]
 56
 57---
 58apiVersion: rbac.authorization.k8s.io/v1
 59kind: ClusterRoleBinding
 60metadata:
 61  name: cluster-autoscaler
 62roleRef:
 63  apiGroup: rbac.authorization.k8s.io
 64  kind: ClusterRole
 65  name: cluster-autoscaler
 66subjects:
 67- kind: ServiceAccount
 68  name: cluster-autoscaler
 69  namespace: kube-system
 70
 71---
 72apiVersion: apps/v1
 73kind: Deployment
 74metadata:
 75  name: cluster-autoscaler
 76  namespace: kube-system
 77  labels:
 78    app: cluster-autoscaler
 79spec:
 80  replicas: 1
 81  selector:
 82    matchLabels:
 83      app: cluster-autoscaler
 84  template:
 85    metadata:
 86      labels:
 87        app: cluster-autoscaler
 88      annotations:
 89        prometheus.io/scrape: "true"
 90        prometheus.io/port: "8085"
 91    spec:
 92      priorityClassName: system-cluster-critical
 93      serviceAccountName: cluster-autoscaler
 94      containers:
 95      # The CA minor version must match the cluster minor version (e.g. v1.28.x for EKS 1.28)
 96      - image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.28.2
 97        name: cluster-autoscaler
 98        resources:
 99          limits:
100            cpu: 100m
101            memory: 600Mi
102          requests:
103            cpu: 100m
104            memory: 600Mi
105        command:
106        - ./cluster-autoscaler
107        - --v=4
108        - --stderrthreshold=info
109        - --cloud-provider=aws
110        - --skip-nodes-with-local-storage=false
111        - --expander=least-waste
112        - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster-name
113        - --balance-similar-node-groups
114        - --skip-nodes-with-system-pods=false
115        # Scale-down configuration
116        - --scale-down-enabled=true
117        - --scale-down-delay-after-add=10m
118        - --scale-down-unneeded-time=10m
119        - --scale-down-utilization-threshold=0.5
120        # Advanced options
121        - --max-node-provision-time=15m
122        - --max-graceful-termination-sec=600
123        - --max-empty-bulk-delete=10
124        - --max-total-unready-percentage=45
125        - --ok-total-unready-count=3
126        - --new-pod-scale-up-delay=0s
127        env:
128        - name: AWS_REGION
129          value: us-west-2
130        volumeMounts:
131        - name: ssl-certs
132          mountPath: /etc/ssl/certs/ca-certificates.crt
133          readOnly: true
134      volumes:
135      - name: ssl-certs
136        hostPath:
137          path: /etc/ssl/certs/ca-bundle.crt

Configuration Options Explained

Expander Strategies:

Expander	Selection Logic	Use Case
least-waste	Minimize unused resources	Cost optimization
most-pods	Fit most pending pods	High pod density
priority	User-defined priorities	Multi-tier workloads
random	Random selection	Testing/development
price	Lowest cost nodes	Budget-constrained

Scale-Down Configuration:

1# Conservative scale-down (production)
2--scale-down-delay-after-add=15m        # Wait 15 min after scale-up
3--scale-down-unneeded-time=20m          # Node idle for 20 min
4--scale-down-utilization-threshold=0.5  # Below 50% utilization
5
6# Aggressive scale-down (dev/staging)
7--scale-down-delay-after-add=5m
8--scale-down-unneeded-time=5m
9--scale-down-utilization-threshold=0.3   # Below 30% utilization

Advanced: Multi-Node Group Configuration

 1# Multiple node groups with different characteristics
 2command:
 3- ./cluster-autoscaler
 4- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
 5
 6# Manual node group specification
 7- --nodes=1:10:my-cluster-general-asg      # General purpose
 8- --nodes=0:20:my-cluster-spot-asg         # Spot instances
 9- --nodes=0:5:my-cluster-gpu-asg           # GPU nodes
10- --nodes=2:8:my-cluster-memory-asg        # Memory-optimized
11
12# Priority-based expander configuration
13---
14apiVersion: v1
15kind: ConfigMap
16metadata:
17  name: cluster-autoscaler-priority-expander
18  namespace: kube-system
19data:
20  priorities: |
21    10:
22      - .*-spot-.*        # Prefer spot instances
23    50:
24      - .*-general-.*     # Then general purpose
25    100:
26      - .*-gpu-.*         # GPU nodes last resort

Preventing Unwanted Scale-Down

Node Annotations:

1# Prevent node from being scaled down
2kubectl annotate node ip-10-0-1-234.ec2.internal \
3  cluster-autoscaler.kubernetes.io/scale-down-disabled=true
4
5# Allow scale-down again
6kubectl annotate node ip-10-0-1-234.ec2.internal \
7  cluster-autoscaler.kubernetes.io/scale-down-disabled-

Pod Annotations:

 1apiVersion: v1
 2kind: Pod
 3metadata:
 4  name: critical-pod
 5  annotations:
 6    # Prevent node with this pod from scaling down
 7    cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
 8spec:
 9  containers:
10  - name: app
11    image: myapp:v1.0

Pros and Cons

Advantages:

Benefit	Description	Value
Mature & Stable	5+ years production use	Battle-tested reliability
Cloud-Agnostic	Works on all major clouds	Portability across providers
Active Community	Official CNCF project	Regular updates, wide support
Cost Optimization	Automatic scale-down	40-60% infrastructure savings
PDB Awareness	Respects disruption budgets	Safe scaling operations

Limitations:

Challenge	Impact	Mitigation
Slow Provisioning	2-5 min node startup	Use warm pools, overprovisioning
ASG-Based	Rigid node group structure	Use Karpenter for flexibility
Limited Intelligence	Basic bin-packing	Priority expander for multi-tier
Scale-Down Delays	Capacity retained longer	Tune thresholds for workload
Node Group Fragmentation	Many ASGs to manage	Consolidate where possible

When to Use Cluster Autoscaler

Ideal Scenarios:

Traditional Kubernetes Clusters (self-managed or early EKS/GKE)
Regulated Environments requiring stable, proven technology
Multi-Cloud Deployments needing consistent behavior
Existing ASG Infrastructure already in place

Not Recommended For:

Highly Dynamic Workloads → Use Karpenter
Spot-Heavy Strategies → Karpenter better handles interruptions
Complex Scheduling Requirements → Karpenter’s just-in-time provisioning

Monitoring Cluster Autoscaler

 1# Prometheus metrics scraping
 2apiVersion: v1
 3kind: Service
 4metadata:
 5  name: cluster-autoscaler
 6  namespace: kube-system
 7  labels:
 8    app: cluster-autoscaler
 9spec:
10  ports:
11  - port: 8085
12    protocol: TCP
13    targetPort: 8085
14    name: metrics
15  selector:
16    app: cluster-autoscaler
17
18---
19# ServiceMonitor for Prometheus Operator
20apiVersion: monitoring.coreos.com/v1
21kind: ServiceMonitor
22metadata:
23  name: cluster-autoscaler
24  namespace: kube-system
25spec:
26  selector:
27    matchLabels:
28      app: cluster-autoscaler
29  endpoints:
30  - port: metrics
31    interval: 30s

Key Metrics:

 1# Cluster Autoscaler specific metrics
 2cluster_autoscaler_scaled_up_nodes_total
 3cluster_autoscaler_scaled_down_nodes_total
 4cluster_autoscaler_unschedulable_pods_count
 5cluster_autoscaler_nodes_count
 6cluster_autoscaler_failed_scale_ups_total
 7
 8# Alert examples
 9- alert: ClusterAutoscalerErrors
10  expr: rate(cluster_autoscaler_errors_total[15m]) > 0
11  for: 15m
12  annotations:
13    summary: "Cluster Autoscaler experiencing errors"
14
15- alert: UnschedulablePods
16  expr: cluster_autoscaler_unschedulable_pods_count > 0
17  for: 10m
18  annotations:
19    summary: "{{ $value }} pods unable to schedule"

Approach 2: Karpenter (Next-Generation Cluster Autoscaling)

Overview and Architecture

Karpenter is a modern, high-performance Kubernetes cluster autoscaler created by AWS that provisions just-in-time compute resources directly without relying on node groups. It represents a paradigm shift in cluster autoscaling.

Karpenter vs Cluster Autoscaler:

CLUSTER AUTOSCALER APPROACH:
┌─────────────────────────────────────────────────────┐
│  Pending Pod → Check ASGs → Select ASG → Scale ASG │
│      ↓             ↓            ↓           ↓       │
│  Fixed      Pre-defined    Limited     Slow (3-5    │
│  Node Types   Configs      Choices      minutes)    │
└─────────────────────────────────────────────────────┘

KARPENTER APPROACH:
┌─────────────────────────────────────────────────────┐
│  Pending Pod → Analyze Needs → Provision Exactly    │
│      ↓              ↓               ↓                │
│  Dynamic      Pod Requests    Right-sized           │
│  Selection    Constraints     Node (30-60s)         │
└─────────────────────────────────────────────────────┘

Key Innovations:

Just-in-Time Provisioning: Creates nodes tailored to pending pods
No Node Groups: Direct EC2 API interaction
Bin-Packing Optimization: Intelligent consolidation
Fast Provisioning: 30-60 second node startup
Spot Optimization: Intelligent diversification

Architecture Overview

┌──────────────────────────────────────────────────────────────────┐
│                    KARPENTER ARCHITECTURE                       │
│                                                                  │
│  ┌────────────────┐        ┌────────────────┐                  │
│  │  KARPENTER     │        │  PROVISIONER   │                  │
│  │  CONTROLLER    │───────▶│  RESOURCES     │                  │
│  │                │        │  (CRDs)        │                  │
│  │ • Watch Pods   │        │                │                  │
│  │ • Scheduling   │        │ • NodePool     │                  │
│  │ • Bin-packing  │        │ • EC2NodeClass │                  │
│  └────────────────┘        └────────────────┘                  │
│         ↓                          ↓                            │
│  ┌────────────────────────────────────────┐                    │
│  │     DECISION ENGINE                     │                    │
│  │                                         │                    │
│  │  1. Analyze pending pod requirements   │                    │
│  │  2. Calculate optimal instance types   │                    │
│  │  3. Check spot/on-demand availability  │                    │
│  │  4. Provision via EC2 API              │                    │
│  │  5. Register node to cluster           │                    │
│  └────────────────────────────────────────┘                    │
│         ↓                                                       │
│  ┌────────────────────────────────────────┐                    │
│  │     CONSOLIDATION ENGINE                │                    │
│  │                                         │                    │
│  │  • Continuously analyze utilization    │                    │
│  │  • Replace with cheaper instances      │                    │
│  │  • Bin-pack to fewer nodes             │                    │
│  │  • Handle spot interruptions           │                    │
│  └────────────────────────────────────────┘                    │
└──────────────────────────────────────────────────────────────────┘

Implementation: Karpenter on EKS

Step 1: Prerequisites and IAM Setup

 1# Set environment variables
 2export CLUSTER_NAME=my-eks-cluster
 3export AWS_REGION=us-west-2
 4export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
 5# Karpenter v1 API (GA since Aug 2024). Pin the latest 1.x release from
 6# https://github.com/aws/karpenter-provider-aws/releases
 7export KARPENTER_VERSION="<latest-1.x-release>"
 8
 9# Create Karpenter IAM role
10cat <<EOF > karpenter-controller-trust-policy.json
11{
12  "Version": "2012-10-17",
13  "Statement": [
14    {
15      "Effect": "Allow",
16      "Principal": {
17        "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/oidc.eks.${AWS_REGION}.amazonaws.com/id/OIDC_ID"
18      },
19      "Action": "sts:AssumeRoleWithWebIdentity",
20      "Condition": {
21        "StringEquals": {
22          "oidc.eks.${AWS_REGION}.amazonaws.com/id/OIDC_ID:aud": "sts.amazonaws.com",
23          "oidc.eks.${AWS_REGION}.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:karpenter:karpenter"
24        }
25      }
26    }
27  ]
28}
29EOF
30
31# Create IAM role
32aws iam create-role \
33  --role-name KarpenterControllerRole-${CLUSTER_NAME} \
34  --assume-role-policy-document file://karpenter-controller-trust-policy.json
35
36# Attach policies
37aws iam attach-role-policy \
38  --role-name KarpenterControllerRole-${CLUSTER_NAME} \
39  --policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/KarpenterControllerPolicy

Step 2: Install Karpenter via Helm

 1# Karpenter v1 charts are published to the public ECR OCI registry
 2# (the old https://charts.karpenter.sh repo only hosts pre-v0.17 charts)
 3helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter \
 4  --namespace karpenter \
 5  --create-namespace \
 6  --version ${KARPENTER_VERSION} \
 7  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterControllerRole-${CLUSTER_NAME} \
 8  --set settings.clusterName=${CLUSTER_NAME} \
 9  --set settings.interruptionQueue=${CLUSTER_NAME} \
10  --set controller.resources.requests.cpu=1 \
11  --set controller.resources.requests.memory=1Gi \
12  --set controller.resources.limits.cpu=1 \
13  --set controller.resources.limits.memory=1Gi \
14  --wait

Step 3: Create NodePool Configuration

  1apiVersion: karpenter.sh/v1
  2kind: NodePool
  3metadata:
  4  name: default
  5spec:
  6  # Template for nodes
  7  template:
  8    metadata:
  9      labels:
 10        workload-type: general
 11    spec:
 12      # Requirements for node selection
 13      requirements:
 14      - key: karpenter.sh/capacity-type
 15        operator: In
 16        values: ["spot", "on-demand"]
 17      - key: kubernetes.io/arch
 18        operator: In
 19        values: ["amd64"]
 20      - key: karpenter.k8s.aws/instance-category
 21        operator: In
 22        values: ["c", "m", "r"]
 23      - key: karpenter.k8s.aws/instance-generation
 24        operator: Gt
 25        values: ["5"]
 26
 27      # Node configuration (v1: group/kind/name are all required)
 28      nodeClassRef:
 29        group: karpenter.k8s.aws
 30        kind: EC2NodeClass
 31        name: default
 32
 33      # Taints for specialized workloads
 34      taints: []
 35
 36      # v1: expireAfter moved from spec.disruption to spec.template.spec
 37      expireAfter: 720h  # 30 days
 38
 39  # Limits for this NodePool
 40  limits:
 41    cpu: "1000"
 42    memory: 1000Gi
 43
 44  # Disruption budget (v1 renamed WhenUnderutilized -> WhenEmptyOrUnderutilized)
 45  disruption:
 46    consolidationPolicy: WhenEmptyOrUnderutilized
 47    consolidateAfter: 1m
 48
 49---
 50apiVersion: karpenter.k8s.aws/v1
 51kind: EC2NodeClass
 52metadata:
 53  name: default
 54spec:
 55  # AMI selection (v1: amiSelectorTerms is required; an alias pins the AMI family)
 56  amiSelectorTerms:
 57  - alias: al2023@latest
 58
 59  # v1: kubelet settings moved from the NodePool to the EC2NodeClass
 60  kubelet:
 61    clusterDNS: ["10.100.0.10"]
 62    maxPods: 110
 63
 64  # Subnet discovery
 65  subnetSelectorTerms:
 66  - tags:
 67      karpenter.sh/discovery: ${CLUSTER_NAME}
 68
 69  # Security group discovery
 70  securityGroupSelectorTerms:
 71  - tags:
 72      karpenter.sh/discovery: ${CLUSTER_NAME}
 73
 74  # IAM instance profile
 75  instanceProfile: KarpenterNodeInstanceProfile-${CLUSTER_NAME}
 76
 77  # No custom userData needed: Karpenter generates the AL2023 (nodeadm)
 78  # bootstrap config itself. /etc/eks/bootstrap.sh only exists on AL2 AMIs.
 79
 80  # Block device mappings
 81  blockDeviceMappings:
 82  - deviceName: /dev/xvda
 83    ebs:
 84      volumeSize: 50Gi
 85      volumeType: gp3
 86      encrypted: true
 87      deleteOnTermination: true
 88
 89  # Metadata options
 90  metadataOptions:
 91    httpEndpoint: enabled
 92    httpProtocolIPv6: disabled
 93    httpPutResponseHopLimit: 2
 94    httpTokens: required
 95
 96  # Tags applied to EC2 instances
 97  tags:
 98    Team: platform
 99    Environment: production
100    ManagedBy: karpenter

Advanced: Multi-NodePool Strategy

Production-Ready Multi-Tier Configuration:

  1# General purpose workloads (spot-optimized)
  2apiVersion: karpenter.sh/v1
  3kind: NodePool
  4metadata:
  5  name: general-spot
  6spec:
  7  template:
  8    metadata:
  9      labels:
 10        workload-type: general
 11        capacity-type: spot
 12    spec:
 13      requirements:
 14      - key: karpenter.sh/capacity-type
 15        operator: In
 16        values: ["spot"]
 17      - key: karpenter.k8s.aws/instance-category
 18        operator: In
 19        values: ["c", "m", "r"]
 20      - key: karpenter.k8s.aws/instance-cpu
 21        operator: In
 22        values: ["4", "8", "16"]
 23      - key: karpenter.k8s.aws/instance-generation
 24        operator: Gt
 25        values: ["5"]
 26      nodeClassRef:
 27        group: karpenter.k8s.aws
 28        kind: EC2NodeClass
 29        name: general
 30
 31  limits:
 32    cpu: "500"
 33    memory: 500Gi
 34
 35  disruption:
 36    consolidationPolicy: WhenEmptyOrUnderutilized
 37    consolidateAfter: 30s
 38
 39---
 40# On-demand for critical workloads
 41apiVersion: karpenter.sh/v1
 42kind: NodePool
 43metadata:
 44  name: critical-ondemand
 45spec:
 46  template:
 47    metadata:
 48      labels:
 49        workload-type: critical
 50        capacity-type: on-demand
 51    spec:
 52      requirements:
 53      - key: karpenter.sh/capacity-type
 54        operator: In
 55        values: ["on-demand"]
 56      - key: karpenter.k8s.aws/instance-category
 57        operator: In
 58        values: ["c", "m"]
 59      - key: karpenter.k8s.aws/instance-size
 60        operator: In
 61        values: ["large", "xlarge", "2xlarge"]
 62      nodeClassRef:
 63        group: karpenter.k8s.aws
 64        kind: EC2NodeClass
 65        name: general
 66      taints:
 67      - key: workload
 68        value: critical
 69        effect: NoSchedule
 70
 71  weight: 50  # Higher priority than spot
 72
 73  limits:
 74    cpu: "200"
 75
 76  disruption:
 77    consolidationPolicy: WhenEmpty
 78    consolidateAfter: 300s
 79
 80---
 81# GPU workloads
 82apiVersion: karpenter.sh/v1
 83kind: NodePool
 84metadata:
 85  name: gpu
 86spec:
 87  template:
 88    metadata:
 89      labels:
 90        workload-type: gpu
 91        nvidia.com/gpu: "true"
 92    spec:
 93      requirements:
 94      - key: karpenter.sh/capacity-type
 95        operator: In
 96        values: ["on-demand", "spot"]
 97      - key: karpenter.k8s.aws/instance-family
 98        operator: In
 99        values: ["p3", "p4", "g5"]
100      - key: node.kubernetes.io/instance-type
101        operator: In
102        values: ["p3.2xlarge", "g5.xlarge", "g5.2xlarge"]
103      nodeClassRef:
104        group: karpenter.k8s.aws
105        kind: EC2NodeClass
106        name: gpu
107      taints:
108      - key: nvidia.com/gpu
109        value: "true"
110        effect: NoSchedule
111      # v1: maxPods now lives in the "gpu" EC2NodeClass (spec.kubelet.maxPods: 50)
112
113  limits:
114    cpu: "100"
115    nvidia.com/gpu: "16"
116
117  disruption:
118    consolidationPolicy: WhenEmpty
119    consolidateAfter: 600s
120
121---
122# Memory-optimized for caching/databases
123apiVersion: karpenter.sh/v1
124kind: NodePool
125metadata:
126  name: memory-optimized
127spec:
128  template:
129    metadata:
130      labels:
131        workload-type: memory-intensive
132    spec:
133      requirements:
134      - key: karpenter.k8s.aws/instance-category
135        operator: In
136        values: ["r", "x"]
137      - key: karpenter.k8s.aws/instance-memory
138        operator: Gt
139        values: ["32768"]  # > 32GB RAM
140      nodeClassRef:
141        group: karpenter.k8s.aws
142        kind: EC2NodeClass
143        name: general
144      taints:
145      - key: workload
146        value: memory-intensive
147        effect: NoSchedule
148
149  limits:
150    memory: 1000Gi
151
152  disruption:
153    consolidationPolicy: WhenEmptyOrUnderutilized
154    consolidateAfter: 300s

Pod Configuration for Karpenter

Using NodePools Effectively:

 1apiVersion: apps/v1
 2kind: Deployment
 3metadata:
 4  name: web-app
 5spec:
 6  replicas: 10
 7  template:
 8    spec:
 9      # Select spot nodes
10      nodeSelector:
11        karpenter.sh/capacity-type: spot
12        workload-type: general
13
14      # No toleration needed for spot interruptions: Karpenter taints the node
15      # (karpenter.sh/disrupted:NoSchedule in v1) and drains it gracefully.
16      # Do NOT tolerate that taint, or pods get rescheduled onto nodes being removed.
17
18      containers:
19      - name: app
20        image: myapp:v1.0
21        resources:
22          requests:
23            cpu: "500m"
24            memory: "512Mi"
25          limits:
26            cpu: "1000m"
27            memory: "1Gi"
28
29---
30# Critical database workload
31apiVersion: apps/v1
32kind: StatefulSet
33metadata:
34  name: database
35spec:
36  replicas: 3
37  template:
38    spec:
39      # Force on-demand nodes
40      nodeSelector:
41        karpenter.sh/capacity-type: on-demand
42        workload-type: critical
43
44      # Require critical node pool
45      tolerations:
46      - key: workload
47        value: critical
48        effect: NoSchedule
49
50      affinity:
51        # Spread across availability zones
52        podAntiAffinity:
53          requiredDuringSchedulingIgnoredDuringExecution:
54          - labelSelector:
55              matchExpressions:
56              - key: app
57                operator: In
58                values: ["database"]
59            topologyKey: topology.kubernetes.io/zone
60
61      containers:
62      - name: postgres
63        image: postgres:14
64        resources:
65          requests:
66            cpu: "4000m"
67            memory: "16Gi"

Karpenter Best Practices

1. Consolidation Configuration:

 1# Aggressive consolidation (cost-optimized)
 2disruption:
 3  consolidationPolicy: WhenEmptyOrUnderutilized
 4  consolidateAfter: 30s
 5
 6# Conservative consolidation (stability-focused)
 7disruption:
 8  consolidationPolicy: WhenEmpty
 9  consolidateAfter: 600s
10
11# Disabled consolidation (manual control)
12disruption:
13  consolidationPolicy: WhenEmpty
14  consolidateAfter: Never

2. Spot Interruption Handling:

1# Karpenter automatically handles spot interruptions once it is pointed at
2# an SQS interruption queue. Since v0.32 the karpenter-global-settings
3# ConfigMap is gone; settings are Helm values / env vars on the controller.
4# Drift detection is always on in v1 (no feature gate).
5settings:
6  clusterName: ${CLUSTER_NAME}
7  # AWS SQS queue for spot interruption notifications
8  interruptionQueue: ${CLUSTER_NAME}

3. Instance Diversification:

 1requirements:
 2# Allow many instance types for better spot availability
 3- key: karpenter.k8s.aws/instance-category
 4  operator: In
 5  values: ["c", "m", "r"]
 6- key: karpenter.k8s.aws/instance-generation
 7  operator: Gt
 8  values: ["5"]  # Only use generation 6+
 9- key: karpenter.k8s.aws/instance-size
10  operator: In
11  values: ["large", "xlarge", "2xlarge", "4xlarge"]

Pros and Cons

Advantages:

Benefit	Description	Impact
Fast Provisioning	30-60s vs 3-5min	5x faster scale-out
Cost Optimization	Right-sized nodes	20-40% additional savings
No Node Groups	Direct EC2 API	Simplified management
Intelligent Consolidation	Automatic bin-packing	Continuous optimization
Spot Optimization	Diversification + handling	70-90% cost reduction
Just-in-Time	Provisions exact needs	Eliminates waste

Limitations:

Challenge	Impact	Consideration
AWS-Specific	EKS only (currently)	Not portable to other clouds
Newer Technology	Less battle-tested	Thorough testing required
Complexity	More configuration options	Learning curve
Breaking Changes	Rapid API evolution	Stay updated on versions

When to Use Karpenter

Ideal Scenarios:

AWS EKS Clusters (native integration)
Highly Dynamic Workloads with variable requirements
Spot-Heavy Strategies needing intelligent diversification
Cost Optimization Focus as primary driver
Modern Architectures embracing latest technologies

Migration Path from Cluster Autoscaler:

1# Phase 1: Deploy Karpenter alongside Cluster Autoscaler
2# Phase 2: Create NodePools for new workloads
3# Phase 3: Gradually migrate workloads to Karpenter nodes
4# Phase 4: Scale down old ASGs
5# Phase 5: Remove Cluster Autoscaler
6
7# Coexistence example
8kubectl label nodes -l eks.amazonaws.com/nodegroup=old-ng \
9  karpenter.sh/managed=false

Monitoring Karpenter

 1# Prometheus metrics
 2apiVersion: v1
 3kind: Service
 4metadata:
 5  name: karpenter-metrics
 6  namespace: karpenter
 7spec:
 8  selector:
 9    app.kubernetes.io/name: karpenter
10  ports:
11  - port: 8080
12    name: metrics
13
14---
15# Key Karpenter metrics
16karpenter_nodes_created
17karpenter_nodes_terminated
18karpenter_pods_state
19karpenter_disruption_decisions_total
20karpenter_interruption_received_messages
21
22# Grafana dashboard
23# https://github.com/aws/karpenter/tree/main/website/content/en/preview/getting-started/getting-started-with-karpenter/grafana-dashboard

Approach 3: AWS EKS-Specific Autoscaling

Managed Node Groups Autoscaling

Native EKS Integration:

 1// AWS CDK example
 2import * as eks from 'aws-cdk-lib/aws-eks';
 3import * as ec2 from 'aws-cdk-lib/aws-ec2';
 4
 5// Create managed node group with autoscaling
 6const nodeGroup = cluster.addNodegroupCapacity('standard-nodes', {
 7  instanceTypes: [
 8    ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.LARGE),
 9    ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
10  ],
11  minSize: 2,
12  maxSize: 20,
13  desiredSize: 5,
14
15  // Spot instances
16  capacityType: eks.CapacityType.SPOT,
17
18  // Scaling configuration
19  amiType: eks.NodegroupAmiType.AL2_X86_64,
20  diskSize: 50,
21
22  // Labels and taints
23  labels: {
24    'workload-type': 'general',
25  },
26
27  // Remote access
28  remoteAccess: {
29    sshKeyName: 'my-key',
30  },
31});

EKS Auto Mode

EKS Auto Mode has been generally available since December 2024 (it was in preview when this section was first drafted). It runs a managed Karpenter for you, so treat it as a first-class alternative to self-managed Cluster Autoscaler or Karpenter.

Fully Managed Compute:

 1# EKS Auto Mode removes need for node management entirely
 2# AWS manages:
 3# - Node provisioning
 4# - Auto-scaling
 5# - Security patching
 6# - Capacity optimization
 7
 8# Enable during cluster creation
 9# (Auto Mode requires compute, block storage and load balancing to be enabled together;
10#  role/subnet flags omitted for brevity)
11aws eks create-cluster \
12  --name my-cluster \
13  --compute-config enabled=true \
14  --kubernetes-network-config '{"elasticLoadBalancing":{"enabled":true}}' \
15  --storage-config '{"blockStorage":{"enabled":true}}'
16
17# Workload specifications drive capacity
18apiVersion: apps/v1
19kind: Deployment
20metadata:
21  name: app
22spec:
23  replicas: 10
24  template:
25    spec:
26      containers:
27      - name: app
28        resources:
29          requests:
30            cpu: "1000m"
31            memory: "2Gi"
32      # EKS Auto Mode handles the rest

AWS Fargate for EKS

Serverless Kubernetes:

 1# Fargate profile
 2apiVersion: v1
 3kind: ConfigMap
 4metadata:
 5  name: fargate-profile
 6data:
 7  profile: |
 8    {
 9      "fargateProfileName": "serverless-apps",
10      "selectors": [
11        {
12          "namespace": "serverless",
13          "labels": {
14            "compute-type": "fargate"
15          }
16        }
17      ]
18    }    
19
20---
21# Pods automatically run on Fargate
22apiVersion: v1
23kind: Pod
24metadata:
25  name: serverless-app
26  namespace: serverless
27  labels:
28    compute-type: fargate
29spec:
30  containers:
31  - name: app
32    image: myapp:v1.0
33    resources:
34      requests:
35        cpu: "500m"
36        memory: "1Gi"
37# No node management needed!

Fargate Pricing Model:

Cost = (vCPU × $0.04048/hour) + (GB RAM × $0.004445/hour)

Example:
2 vCPU + 4GB RAM = (2 × $0.04048) + (4 × $0.004445)
                 = $0.08096 + $0.01778
                 = $0.09874 per hour
                 = $71/month (24/7)

vs EC2 t3.medium (2vCPU, 4GB) = $30/month

Fargate Cost-Effective When:
- Intermittent workloads (not 24/7)
- Need zero operational overhead
- Compliance/isolation requirements

Approach 4: GKE-Specific Autoscaling

GKE Cluster Autoscaler

Native GKE Integration:

 1# GKE cluster with autoscaling
 2gcloud container clusters create my-cluster \
 3  --enable-autoscaling \
 4  --min-nodes=1 \
 5  --max-nodes=10 \
 6  --zone=us-central1-a \
 7  --machine-type=n1-standard-4 \
 8  --enable-autoprovisioning \
 9  --min-cpu=1 \
10  --max-cpu=100 \
11  --min-memory=1 \
12  --max-memory=1000 \
13  --autoprovisioning-scopes=https://www.googleapis.com/auth/compute

Node Auto-Provisioning (NAP)

Intelligent Node Pool Creation:

 1# GKE automatically creates node pools based on workload needs
 2gcloud container clusters update my-cluster \
 3  --enable-autoprovisioning \
 4  --autoprovisioning-config-file=config.yaml
 5
 6# config.yaml
 7resourceLimits:
 8- resourceType: cpu
 9  minimum: 1
10  maximum: 100
11- resourceType: memory
12  minimum: 1
13  maximum: 1000
14- resourceType: nvidia-tesla-k80
15  minimum: 0
16  maximum: 4
17
18autoscalingProfile: OPTIMIZE_UTILIZATION  # or BALANCED
19
20management:
21  autoUpgrade: true
22  autoRepair: true

How NAP Works:

Pod with GPU → No suitable node → NAP creates GPU node pool → Pod schedules
     ↓                ↓                     ↓                      ↓
Specific     Analyze pod        Choose optimal          Auto-scale
Requirements  requirements      instance type           as needed

GKE Autopilot

Fully Managed GKE:

 1# Create Autopilot cluster
 2gcloud container clusters create-auto my-autopilot-cluster \
 3  --region=us-central1
 4
 5# Autopilot handles:
 6# - Node provisioning
 7# - Auto-scaling
 8# - Security hardening
 9# - Capacity optimization
10# - Networking configuration
11
12# You only manage workloads
13kubectl apply -f deployment.yaml
14
15# Autopilot automatically:
16# - Provisions right-sized nodes
17# - Scales based on pod needs
18# - Optimizes cost and performance
19# - Handles node upgrades

Autopilot Pricing:

Cost = Sum of pod resource requests

Example Deployment:
10 pods × (0.5 vCPU + 1GB RAM)
= 5 vCPU + 10GB RAM
= (5 × $0.04208) + (10 × $0.00463)
= $0.2104 + $0.0463
= $0.2567 per hour
= $185/month

Includes:
- Compute resources
- GKE management fee
- Networking egress (within limits)

Pros and Cons

GKE Autoscaling Advantages:

Feature	Benefit
Node Auto-Provisioning	Creates optimal node pools automatically
Autopilot Mode	Zero node management
Integrated Monitoring	Built-in Cloud Monitoring
Fast Provisioning	GCE startup optimization
Preemptible VM Support	80% cost savings

Limitations:

Challenge	Impact
GCP Lock-in	Not portable
Autopilot Constraints	Limited customization
Cost	Premium pricing for convenience

Approach 5: Azure AKS-Specific Autoscaling

AKS Cluster Autoscaler

 1# Enable cluster autoscaler
 2az aks update \
 3  --resource-group myResourceGroup \
 4  --name myAKSCluster \
 5  --enable-cluster-autoscaler \
 6  --min-count 1 \
 7  --max-count 10
 8
 9# Multiple node pools
10az aks nodepool add \
11  --resource-group myResourceGroup \
12  --cluster-name myAKSCluster \
13  --name spotpool \
14  --enable-cluster-autoscaler \
15  --min-count 0 \
16  --max-count 20 \
17  --priority Spot \
18  --eviction-policy Delete \
19  --spot-max-price -1 \
20  --node-vm-size Standard_DS2_v2

Azure Container Instances (ACI) Integration

Virtual Nodes (Serverless):

 1# Enable virtual nodes
 2az aks enable-addons \
 3  --resource-group myResourceGroup \
 4  --name myAKSCluster \
 5  --addons virtual-node \
 6  --subnet-name VirtualNodeSubnet
 7
 8# Pods with virtual-kubelet toleration run on ACI
 9apiVersion: v1
10kind: Pod
11metadata:
12  name: serverless-pod
13spec:
14  containers:
15  - name: app
16    image: myapp:v1.0
17  tolerations:
18  - key: virtual-kubelet.io/provider
19    operator: Equal
20    value: azure
21    effect: NoSchedule
22  nodeSelector:
23    type: virtual-kubelet

Comparison: Cloud Provider Autoscaling Solutions

Feature	EKS	GKE	AKS
Cluster Autoscaler	✅ Standard	✅ Standard	✅ Standard
Advanced Autoscaler	Karpenter	NAP	Standard CA
Serverless Pods	Fargate	Autopilot	ACI Virtual Nodes
Fully Managed	EKS Auto Mode	Autopilot	AKS Automatic
Spot Instance Support	✅ Excellent	✅ Preemptible	✅ Spot VMs
Provisioning Speed	2-5 min (30s Karpenter)	1-3 min	2-4 min
Cost Optimization	Karpenter best-in-class	NAP intelligent	Standard
Multi-Architecture	✅ ARM64 support	✅ ARM64 support	Limited

Emerging Autoscaling Technologies

1. Kamaji (Multi-Tenant Control Planes)

 1# Virtual control plane per tenant
 2apiVersion: kamaji.clastix.io/v1alpha1
 3kind: TenantControlPlane
 4metadata:
 5  name: tenant-a
 6spec:
 7  controlPlane:
 8    deployment:
 9      replicas: 2
10  network:
11    serviceType: LoadBalancer
12  addons:
13    coreDNS: {}
14    konnectivity: {}
15
16# Each tenant gets isolated autoscaling

2. Kwok (Kubernetes WithOut Kubelet)

1# Simulate thousands of nodes for testing autoscaling
2kwok \
3  --kubeconfig=~/.kube/config \
4  --manage-all-nodes=false \
5  --manage-nodes-with-annotation-selector=kwok.x-k8s.io/node=fake \
6  --disregard-status-with-annotation-selector=kwok.x-k8s.io/status=custom
7
8# Test autoscaling logic without real infrastructure cost

3. Volcano (Batch Job Scheduling)

 1# Advanced scheduling for ML/batch workloads
 2apiVersion: batch.volcano.sh/v1alpha1
 3kind: Job
 4metadata:
 5  name: ml-training
 6spec:
 7  minAvailable: 4
 8  schedulerName: volcano
 9  policies:
10  - event: PodEvicted
11    action: RestartJob
12  tasks:
13  - replicas: 8
14    name: worker
15    template:
16      spec:
17        containers:
18        - name: worker
19          image: ml-trainer:v1.0
20          resources:
21            requests:
22              nvidia.com/gpu: 1
23
24# Volcano coordinates autoscaling with job scheduling

Production Best Practices

1. Hybrid Autoscaling Strategy

 1# Baseline: Cluster Autoscaler for stability
 2# Dynamic: Karpenter for optimization
 3# Serverless: Fargate/Autopilot for burstiness
 4
 5apiVersion: v1
 6kind: ConfigMap
 7metadata:
 8  name: autoscaling-strategy
 9data:
10  strategy: |
11    Tier 1 (Critical): On-demand nodes, Cluster Autoscaler
12    Tier 2 (Standard): Mix spot/on-demand, Karpenter
13    Tier 3 (Batch): Pure spot, Karpenter with aggressive consolidation
14    Tier 4 (Burst): Fargate/Autopilot, scale-to-zero

2. Cost Optimization Tactics

 1# Multi-dimensional cost optimization
 2priorities:
 3  1. Spot instances (70-90% savings)
 4  2. Right-sizing via Karpenter
 5  3. Consolidation during low traffic
 6  4. Reserved instances for baseline
 7  5. Savings Plans for predictable workloads
 8
 9# Example cost breakdown
10baseline: 10 on-demand nodes (reserved) = $1,500/month
11dynamic: 0-50 spot nodes (Karpenter) = $500-3000/month
12burst: Fargate for spikes = $200/month
13Total: $2,200-4,700/month vs $15,000 static
14Savings: 68-85%

3. Monitoring and Alerting

 1# Comprehensive autoscaling observability
 2apiVersion: monitoring.coreos.com/v1
 3kind: PrometheusRule
 4metadata:
 5  name: autoscaling-alerts
 6spec:
 7  groups:
 8  - name: cluster-autoscaling
 9    rules:
10    - alert: ClusterFullCapacity
11      expr: |
12        sum(kube_node_status_allocatable{resource="cpu"})
13        - sum(kube_pod_container_resource_requests{resource="cpu"})
14        < 10        
15      for: 5m
16      annotations:
17        summary: "Cluster near full capacity"
18
19    - alert: HighSpotInterruptionRate
20      expr: rate(karpenter_interruption_received_messages[5m]) > 0.1
21      annotations:
22        summary: "High spot interruption rate"
23
24    - alert: AutoscalingDisabled
25      expr: up{job="cluster-autoscaler"} == 0
26      for: 5m
27      annotations:
28        summary: "Cluster autoscaler is down"
29
30    - alert: NodeProvisioningDelayed
31      expr: |
32        sum(karpenter_pending_pods_total) > 10
33        AND
34        rate(karpenter_nodes_created[5m]) == 0        
35      for: 10m
36      annotations:
37        summary: "Nodes not provisioning despite pending pods"

4. Testing Autoscaling

 1# Load testing script
 2#!/bin/bash
 3
 4# Test scale-up
 5kubectl run load-generator-1 --image=busybox:1.28 \
 6  --restart=Never --rm -i --tty -- /bin/sh -c \
 7  "while true; do wget -q -O- http://test-service; sleep 0.01; done" &
 8
 9# Monitor scaling
10watch -n 5 'kubectl get nodes; kubectl get hpa; kubectl top nodes'
11
12# Test scale-down
13# Stop load and observe consolidation
14
15# Test spot interruption (Karpenter)
16# Manually terminate spot instance to verify graceful handling
17aws ec2 terminate-instances --instance-ids i-xxxxx
18
19# Verify:
20# - New node provisions
21# - Pods reschedule
22# - No downtime

For comprehensive Kubernetes knowledge, explore these related posts:

Horizontal Pod Autoscaling

Part 1: Horizontal Pod Autoscaler - Deep dive into HPA, KEDA, custom metrics, and event-driven autoscaling

Kubernetes Fundamentals

Kubernetes Complete Guide (Part 1): Introduction - Architecture, concepts, installation (Traditional Chinese)
Kubernetes Complete Guide (Part 3): Advanced Features - RBAC, monitoring, production practices (Traditional Chinese)

Production Kubernetes

Building Production Kubernetes Platform on AWS EKS - Complete EKS architecture with CDK implementation

Conclusion

Cluster-level autoscaling has evolved significantly, offering multiple approaches for different needs:

Decision Framework

Choose Cluster Autoscaler when:

Running on any cloud or on-premises
Need stable, proven technology
Existing ASG/node group infrastructure
Regulatory requirements for specific tech

Choose Karpenter when:

On AWS EKS
Cost optimization is critical
Dynamic, unpredictable workloads
Want latest autoscaling capabilities

Choose Cloud Provider Solutions when:

Deep cloud integration needed
Minimal operational overhead desired
Willing to accept vendor lock-in
Budget allows premium pricing

Key Takeaways

Layer Your Autoscaling: Combine pod (HPA) and cluster autoscaling
Start Simple: Begin with Cluster Autoscaler, evolve to Karpenter/cloud solutions
Embrace Spot/Preemptible: 70-90% cost savings possible
Monitor Comprehensively: Autoscaling health is critical
Test Under Load: Validate behavior before production

Future of Kubernetes Autoscaling

The autoscaling landscape continues evolving:

AI-Driven Autoscaling: Predictive scaling using ML models
Multi-Cluster Autoscaling: Federated capacity management
Sustainability-Aware: Carbon-optimized instance selection
FinOps Integration: Real-time cost optimization
Edge Computing: Autoscaling for edge Kubernetes

By understanding the full spectrum of autoscaling approaches—from traditional Cluster Autoscaler to cutting-edge Karpenter and cloud-native solutions—you can architect Kubernetes platforms that automatically adapt to demand while optimizing costs and maintaining reliability.

The future belongs to intelligent, multi-layered autoscaling strategies that combine the best of opensource innovation with cloud provider capabilities, delivering both operational excellence and cost efficiency at scale.

Series Overview

The Complete Autoscaling Picture

Multi-Layer Autoscaling Architecture

Why Cluster Autoscaling Matters

Approach 1: Kubernetes Cluster Autoscaler (CA)

Overview and Architecture

Implementation: Cluster Autoscaler on Self-Managed Kubernetes

Configuration Options Explained

Advanced: Multi-Node Group Configuration

Preventing Unwanted Scale-Down

Pros and Cons

When to Use Cluster Autoscaler

Monitoring Cluster Autoscaler

Approach 2: Karpenter (Next-Generation Cluster Autoscaling)

Overview and Architecture

Architecture Overview

Implementation: Karpenter on EKS

Advanced: Multi-NodePool Strategy

Pod Configuration for Karpenter

Karpenter Best Practices

Pros and Cons

When to Use Karpenter

Monitoring Karpenter

Approach 3: AWS EKS-Specific Autoscaling

Managed Node Groups Autoscaling

EKS Auto Mode

AWS Fargate for EKS

Approach 4: GKE-Specific Autoscaling

GKE Cluster Autoscaler

Node Auto-Provisioning (NAP)

GKE Autopilot

Pros and Cons

Approach 5: Azure AKS-Specific Autoscaling

AKS Cluster Autoscaler

Azure Container Instances (ACI) Integration

Comparison: Cloud Provider Autoscaling Solutions

Emerging Autoscaling Technologies

1. Kamaji (Multi-Tenant Control Planes)

2. Kwok (Kubernetes WithOut Kubelet)

3. Volcano (Batch Job Scheduling)

Production Best Practices

1. Hybrid Autoscaling Strategy

2. Cost Optimization Tactics

3. Monitoring and Alerting

4. Testing Autoscaling

Related Topics

Horizontal Pod Autoscaling

Kubernetes Fundamentals

Production Kubernetes

Conclusion

Decision Framework

Key Takeaways

Future of Kubernetes Autoscaling

Yen

Related Articles

Kubernetes Autoscaling Complete Guide (Part 1): Horizontal Pod Autoscaler

Kubernetes 完整指南（三）：進階功能與生產環境實踐

Kubernetes 完整指南（二）：核心資源與 kubectl 實戰操作