EMR Rules

These rules identify EMR clusters using outdated instance types and clusters sitting idle without processing work.

CLDBRN-AWS-EMR-1

EMR Cluster Previous Generation Instance Types

Scan type: Discovery and IaC

Severity: Medium

What it checks

Flags active EMR clusters that use previous-generation EC2 instance types across any of their instance groups. The rule uses the same preferred instance family classification as the EC2 rules to determine which instance types are current-generation.

Why it matters

Previous-generation instance types cost more per unit of compute than their current-gen replacements. An EMR cluster running m4.xlarge nodes instead of m7g.xlarge pays a higher hourly rate and gets worse performance. For long-running or large clusters, the difference adds up fast.

What triggers a finding

In discovery scans, the cluster is still active (no endDateTime) and any instance type in its instance groups is classified as non-preferred. In IaC scans, any declared instance type classified as non-preferred triggers the rule; there is no runtime state check.

How to remediate

1. Identify which instance groups use previous-gen types
2. Create a new cluster with current-gen instance types, or modify the instance fleet configuration if your cluster uses instance fleets:

aws emr create-cluster \
  --name "my-cluster-upgraded" \
  --instance-groups InstanceGroupType=MASTER,InstanceType=m7g.xlarge,InstanceCount=1 \
                    InstanceGroupType=CORE,InstanceType=m7g.2xlarge,InstanceCount=4 \
  --release-label emr-7.0.0

Graviton-based instances (m7g, r7g, c7g) give you the best price/performance ratio on EMR. Check AWS EMR supported instance types for compatibility with your EMR release.

IaC resources checked

CLDBRN-AWS-EMR-2

EMR Cluster Idle

Scan type: Discovery

Severity: High

What it checks

Flags active EMR clusters whose IsIdle CloudWatch metric has been true for at least 30 minutes. EMR publishes this metric every 5 minutes, so the rule checks for 6 or more idle periods within the last 30 minutes.

Why it matters

EMR clusters bill for every hour their instances run, whether processing jobs or not. A cluster stuck in WAITING state with no work to do is burning money. Development clusters and ad-hoc analytics clusters are common offenders here - they get spun up, the work finishes, and nobody terminates them.

What triggers a finding

All of the following must be true:

• Cluster has no endDateTime (still running)
• Cluster state is RUNNING or WAITING
• The cluster has 6 or more idle periods within the last 30 minutes (30+ minutes of idle time)

Idle metrics are joined to the cluster inventory by account, region, and cluster ID, so a metric series is never matched against a same-named cluster in another account or region.

Coverage

A cluster that is still running in RUNNING or WAITING state but has no usable IsIdle period count is reported as unknown rather than as a pass. See Live Evaluation Coverage.

How to remediate

1. Check if the cluster has pending steps or is expected to receive work soon
2. If the cluster is no longer needed, terminate it:

aws emr terminate-clusters --cluster-ids j-XXXXXXXXXXXXX

3. For clusters that run intermittent workloads, configure auto-termination to shut down after an idle period:

aws emr modify-cluster \
  --cluster-id j-XXXXXXXXXXXXX \
  --auto-terminate-policy IdleTimeout=3600

For recurring jobs, consider using EMR Serverless or EMR on EKS instead. Both eliminate the idle cluster cost problem entirely by only billing for actual compute time.

See Also

• CLI discover command -- scan live EMR clusters
• SDK Reference -- run scans programmatically