tag: Apache-Spark · 4 items
- Platform/SRE — Skip
- CI/CD — Skip
- Leader — Learn: GA feature that eliminates the need for separate data quality infrastructure alongside EMR/Spark workloads; worth noting when evaluating build-vs-buy for data validation tooling at no incremental cost.
- Platform/SRE — Plan: Teams running EMR on EKS should evaluate migrating production Spark workloads to emr-spark-8.1.0 to take advantage of the new 36-month LTS support window, reducing forced upgrade pressure; no hard deadline exists yet.
- CI/CD — Skip
- Leader — Plan: EMR LTS changes the upgrade economics for data platform teams — evaluate standardizing on LTS releases to extend support cycles and reduce upgrade toil as part of the next data platform planning cycle.
- Platform/SRE — Plan: If your platform runs EMR workloads, this GA release brings notable operational improvements: Serverless shuffle storage raised from 200 GiB to 1 TiB, IPv6 EKS cluster support, and Spark Connect endpoints on EMR on EKS. Schedule evaluation for teams hitting storage or networking constraints this quarter.
- CI/CD — Skip
- Leader — Skip
- Platform/SRE — Skip
- CI/CD — Skip
- Leader — Learn: AWS is extending AI-assisted diagnostics across all EMR deployment modes; relevant for leaders evaluating whether AI tooling (including MCP-based agents) changes the skills or support model needed for data platform operations.