These rules identify EMR clusters using outdated instance types and clusters sitting idle without processing work.
| Rule ID | Scan Type | Severity | Name |
|---|---|---|---|
| CLDBRN-AWS-EMR-1 | Discovery and IaC | Medium | EMR Cluster Previous Generation Instance Types |
| CLDBRN-AWS-EMR-2 | Discovery | High | EMR Cluster Idle |
CLDBRN-AWS-EMR-1
EMR Cluster Previous Generation Instance Types
Scan type: Discovery and IaC
Severity: Medium
What it checks
Flags active EMR clusters that use previous-generation EC2 instance types across any of their instance groups. The rule uses the same preferred instance family classification as the EC2 rules to determine which instance types are current-generation.
Why it matters
Previous-generation instance types cost more per unit of compute than their current-gen replacements. An EMR cluster running m4.xlarge nodes instead of m7g.xlarge pays a higher hourly rate and gets worse performance. For long-running or large clusters, the difference adds up fast.
What triggers a finding
The cluster is still active (no endDateTime) and any instance type in its instance groups is classified as non-preferred.
How to remediate
- Identify which instance groups use previous-gen types
- Create a new cluster with current-gen instance types, or modify the instance fleet configuration if your cluster uses instance fleets:
aws emr create-cluster \
--name "my-cluster-upgraded" \
--instance-groups InstanceGroupType=MASTER,InstanceType=m7g.xlarge,InstanceCount=1 \
InstanceGroupType=CORE,InstanceType=m7g.2xlarge,InstanceCount=4 \
--release-label emr-7.0.0
Graviton-based instances (m7g, r7g, c7g) give you the best price/performance ratio on EMR. Check AWS EMR supported instance types for compatibility with your EMR release.
IaC resources checked
| IaC Tool | Resource Type |
|---|---|
| Terraform | aws_emr_cluster |
| CloudFormation | AWS::EMR::Cluster |
CLDBRN-AWS-EMR-2
EMR Cluster Idle
Scan type: Discovery
Severity: High
What it checks
Flags active EMR clusters whose IsIdle CloudWatch metric has been true for at least 30 minutes. EMR publishes this metric every 5 minutes, so the rule checks for 6 or more idle periods within the last 30 minutes.
Why it matters
EMR clusters bill for every hour their instances run, whether processing jobs or not. A cluster stuck in WAITING state with no work to do is burning money. Development clusters and ad-hoc analytics clusters are common offenders here - they get spun up, the work finishes, and nobody terminates them.
What triggers a finding
All of the following must be true:
- Cluster has no
endDateTime(still running) - Cluster state is
RUNNINGorWAITING - The cluster has 6 or more idle periods within the last 30 minutes (30+ minutes of idle time)
Idle metrics are joined to the cluster inventory by account, region, and cluster ID, so a metric series is never matched against a same-named cluster in another account or region.
Coverage
A cluster that is still running in RUNNING or WAITING state but has no usable IsIdle period count is reported as unknown rather than as a pass. See Live Evaluation Coverage.
How to remediate
- Check if the cluster has pending steps or is expected to receive work soon
- If the cluster is no longer needed, terminate it:
aws emr terminate-clusters --cluster-ids j-XXXXXXXXXXXXX
- For clusters that run intermittent workloads, configure auto-termination to shut down after an idle period:
aws emr modify-cluster \
--cluster-id j-XXXXXXXXXXXXX \
--auto-terminate-policy IdleTimeout=3600
For recurring jobs, consider using EMR Serverless or EMR on EKS instead. Both eliminate the idle cluster cost problem entirely by only billing for actual compute time.
See Also
- CLI discover command -- scan live EMR clusters
- SDK Reference -- run scans programmatically