CNCF launches Kubernetes AI Requirements (KARs) in v1.35. Standardizing hardware orchestration for autonomous AI agents at scale. Read the technical breakdown.
What Kubernetes AI Requirements Actually Standardize
Kubernetes v1.35 introduces Kubernetes AI Requirements (KARs), a CNCF conformance effort aimed at agentic ops: clusters that host autonomous AI agents and the hardware those agents depend on. Until now, teams stitched together GPU scheduling, device plugins, memory isolation, and model-serving sidecars with ad hoc labels, custom CRDs, and vendor-specific operators. That worked for a single platform team; it broke down when agents needed portable guarantees about accelerators, topology, and isolation across environments.
KARs do not replace the Kubernetes API. They define a conformance surface—capabilities a cluster must expose and behaviors it must honor—so an agent runtime can discover, request, and release hardware-backed resources without hardcoding a particular cloud or device stack. The practical shift is from “we hope this node has a free accelerator” to “the cluster advertises AI-relevant capacity and enforces allocation rules the same way everywhere.”
Hardware Orchestration for Agents, Not Just Jobs
Batch training jobs and long-running agents place different demands on the control plane. Jobs can tolerate queueing and preemption; agents often hold sessions open, stream tool calls, and need predictable access to accelerators, high-bandwidth memory, and local scratch. KAR-oriented orchestration treats those resources as first-class scheduling inputs: device class, topology affinity, exclusive vs. shared access, and teardown when an agent pod exits or scales down.
At scale, that means the scheduler and device layer must agree on ownership. An agent that claims a GPU fraction must not be silently oversubscribed by another workload. Nodes should report capacity in terms operators and agents can both reason about—not only raw counts, but constraints that matter for inference and multi-model hosts (NUMA placement, peer-to-peer device links, eviction under pressure). Standardization here reduces the custom glue that usually lives only in one team’s Helm charts.
- Advertise AI-capable node and device inventory through stable, discoverable APIs.
- Request and bind hardware with clear exclusivity and topology constraints.
- Release resources reliably when agent workloads terminate or reschedule.
- Keep multi-tenant isolation so one agent cannot starve or observe another’s devices.
What Platform Teams Should Change in Practice
If you already run inference or agent fleets on Kubernetes, treat KARs as a checklist for your control plane rather than a single feature flag. Inventory how you expose GPUs and related devices today; map that path to the conformance expectations for discovery, scheduling hooks, and resource lifecycle. Prefer in-tree or CNCF-aligned device and scheduling patterns over one-off annotations that only your internal orchestrator understands.
For agentic ops specifically, separate “agent identity and policy” from “hardware claim.” Agents still need RBAC, network policy, and secret access; KARs focus on whether the cluster can honestly schedule and isolate the hardware those agents require. When promoting a cluster to production for autonomous workloads, verify that your node pools, device plugins, and autoscaling behave under concurrent agent churn—not only under steady batch load.
Adoption Path Without Betting the Fleet
Start with a non-production pool that claims KAR-aligned capability and run a small set of agent runtimes against it: one that needs exclusive accelerator access, one that shares, and one that fails closed when capacity is gone. Measure whether reschedules, node drains, and scale-down free devices without manual cleanup. Document which labels, resource names, and CRDs you will retire once the conformance surface covers the same needs.
Kubernetes v1.35’s KAR launch is a coordination mechanism more than a single binary change. The value shows up when agent frameworks, platform operators, and hardware vendors agree on the same contract for orchestration. Teams that align early reduce lock-in and avoid rewriting scheduling logic every time they add a new accelerator class or multi-cluster agent mesh.