Overview

EMR Rules

CloudBurn cost optimization rules for AWS EMR.

EMR Rules

These rules identify EMR clusters using outdated instance types and clusters sitting idle without processing work.

Rule IDScan TypeName
CLDBRN-AWS-EMR-1DiscoveryEMR Cluster Previous Generation Instance Types
CLDBRN-AWS-EMR-2DiscoveryEMR Cluster Idle

CLDBRN-AWS-EMR-1

EMR Cluster Previous Generation Instance Types

Scan type: Discovery

What it checks

Flags active EMR clusters that use previous-generation EC2 instance types across any of their instance groups. The rule uses the same preferred instance family classification as the EC2 rules to determine which instance types are current-generation.

Why it matters

Previous-generation instance types cost more per unit of compute than their current-gen replacements. An EMR cluster running m4.xlarge nodes instead of m7g.xlarge pays a higher hourly rate and gets worse performance. For long-running or large clusters, the difference adds up fast.

What triggers a finding

The cluster is still active (no endDateTime) and any instance type in its instance groups is classified as non-preferred.

How to remediate

  1. Identify which instance groups use previous-gen types
  2. Create a new cluster with current-gen instance types, or modify the instance fleet configuration if your cluster uses instance fleets:
aws emr create-cluster \
  --name "my-cluster-upgraded" \
  --instance-groups InstanceGroupType=MASTER,InstanceType=m7g.xlarge,InstanceCount=1 \
                    InstanceGroupType=CORE,InstanceType=m7g.2xlarge,InstanceCount=4 \
  --release-label emr-7.0.0

Graviton-based instances (m7g, r7g, c7g) give you the best price/performance ratio on EMR. Check AWS EMR supported instance types for compatibility with your EMR release.

CLDBRN-AWS-EMR-2

EMR Cluster Idle

Scan type: Discovery

What it checks

Flags active EMR clusters whose IsIdle CloudWatch metric has been true for at least 30 consecutive minutes. EMR publishes this metric every 5 minutes, so the rule checks for 6 consecutive idle periods.

Why it matters

EMR clusters bill for every hour their instances run, whether processing jobs or not. A cluster stuck in WAITING state with no work to do is burning money. Development clusters and ad-hoc analytics clusters are common offenders here - they get spun up, the work finishes, and nobody terminates them.

What triggers a finding

All of the following must be true:

  • Cluster has no endDateTime (still running)
  • Cluster state is RUNNING or WAITING
  • The cluster has been idle for 6 or more consecutive 5-minute periods (30+ minutes)

How to remediate

  1. Check if the cluster has pending steps or is expected to receive work soon
  2. If the cluster is no longer needed, terminate it:
aws emr terminate-clusters --cluster-ids j-XXXXXXXXXXXXX
  1. For clusters that run intermittent workloads, configure auto-termination to shut down after an idle period:
aws emr modify-cluster \
  --cluster-id j-XXXXXXXXXXXXX \
  --auto-terminate-policy IdleTimeout=3600

For recurring jobs, consider using EMR Serverless or EMR on EKS instead. Both eliminate the idle cluster cost problem entirely by only billing for actual compute time.

See Also

← ELBLambda →