ARM64 and AI silicon require precise resource requests to avoid throttling. Learn the exact spec overrides for Graviton4, TPUv5, and H100 nodes. Full breakdown.
Why generic requests fail on specialized silicon
Kubernetes schedules pods based on the CPU and memory numbers you declare. On general-purpose nodes that model is rough but workable. On ARM64 and AI accelerators it is not. Those machines bill and throttle against different units—architecture-specific CPU capacity, device memory, and accelerator cores—so a request copied from an x86 service often lands a pod on hardware it cannot use well, or worse, leaves it runnable while the scheduler thinks the node still has headroom.
The fix is not “request more of everything.” It is to treat each node class as its own capacity model: match the binary and image to the architecture, declare the accelerator resource the device plugin exposes, and size CPU and host memory so the workload is not CPU-starved while waiting on the device or memory-throttled while the accelerator sits idle.
Graviton4 (ARM64): requests that match the architecture
Graviton4 nodes are ARM64. Images built for amd64 will not run unless you multi-arch the image or recompile. Once the pod can start, set CPU and memory requests from measured usage on ARM64, not from an x86 baseline. Instruction mix and memory bandwidth differ; a service that looked underutilized on one architecture can pin cores on the other.
Prefer requests close to steady-state use and limits only where you need a hard ceiling. On dense ARM64 pools, inflated requests waste capacity; missing architecture selectors waste time on FailedScheduling or CrashLoopBackOff. Use node selectors or affinity for ARM64, and keep resource names standard (cpu, memory) unless your platform adds custom metrics for ARM-specific accounting.
TPUv5 and H100: accelerator-aware requests
TPU and GPU nodes do not schedule primarily on millicores. They schedule on device resources—typically a vendor or cloud-specific name that the device plugin advertises. A pod that requests only cpu and memory can land on a GPU or TPU node without claiming the device, which either wastes the machine or leaves the process without the accelerator it expects.
- Request the accelerator resource explicitly (count or fraction as your plugin allows) and pin the pod to the matching node pool.
- Size host CPU and memory for data load, preprocessing, and driver overhead so the device is not stalled by the host.
- Size device memory in the job config or framework flags separately from Kubernetes memory limits; host OOM kills and device OOM are different failure modes.
- Avoid packing unrelated CPU-only work onto accelerator nodes unless you intentionally isolate with taints and tolerations.
For H100-class GPUs, multi-instance or time-sliced sharing only works if every pod’s request matches how the plugin partitions the card. For TPUv5, topology and chip count matter: requesting the wrong shape fails scheduling or forces a fallback topology that breaks training or inference assumptions. Encode those constraints in the pod spec, not only in job scripts.
A practical override pattern
Keep one base Deployment or Job template and override resources per node class with a patch, overlay, or values file: architecture affinity for Graviton4, device requests and taints for TPUv5 and H100, and CPU/memory numbers measured on that class. Document the unit each number maps to—host CPU, host RAM, accelerator count, device memory—so operators do not “optimize” by copying limits across pool types.
Validate with a canary pod that prints architecture, visible devices, and cgroup limits before scaling the real workload. If the scheduler places the pod correctly but the process still throttles, the gap is usually inside the app (batch size, thread pool, or device memory) rather than a missing Kubernetes field. Fix the request model first, then tune the process—so capacity planning and runtime behavior stay aligned.