Agents are sessions, not requests: size GKE node pools for AI agent
Google Cloud outlined a GKE architecture using Agent Substrate 1.0 to park idle AI agent sessions in object storage, cutting Kubernetes node costs by 40%.
By Dillip Chowdary • Nov 16, 2026 • Source: Google Developers Blog

At a glance
- Product
- Agent Substrate 1.0 and AX
- Platform
- Google Kubernetes Engine
- Release date
- November 16, 2026
- Availability
- Generally available on GKE
- Cost reduction
- Up to 40% on Kubernetes node pools
- Break-even fleet
- 37 coding agents or 13 chat assistants
In this post
Google Cloud released a technical framework and cost model detailing how engineering organizations can run persistent coding agent sessions on Google Kubernetes Engine without paying full compute rates for idle time, as detailed in Google Developers Blog's report. The analysis demonstrates that pairing Agent Substrate 1.0 with AX enables clusters to snapshot idle agent workspaces to object storage, cutting Kubernetes node costs by up to 40 percent.
This report examines how session-based agent workloads differ from stateless web traffic, walks through the runtime architecture of Agent Substrate and AX on GKE, and models exact cluster capacity for platform engineers. It provides node pool sizing rules, idle timeout thresholds, and traffic break-even points for teams managing coding agent fleets.
01Agent idle time between developer prompts
Unlike traditional web requests that complete within milliseconds and discard state immediately, AI coding agent sessions remain active across an entire eight-hour workday. A research study of GitHub Copilot agent traces conducted by Banruo Liu and colleagues revealed that an agent waits for developer input roughly 80 percent of the time during typical multi-turn interactions. Active work occurs only in short bursts when processing instructions.
Across those observed traces, a single turn encompassing one user prompt and the agent's complete response required an average of 396.3 seconds of compute time. Assuming a baseline developer cadence of two prompts per hour across an eight-hour shift, the agent performs active computation for only 22 percent of the day. The remaining 78 percent of cluster time is spent idling while awaiting human direction.
02Dedicated pods versus scale to zero
Engineering teams typically evaluate two conventional deployment strategies when migrating agent workloads to Kubernetes clusters. The first strategy assigns an isolated Pod running 1 vCPU and 1 GiB of memory to each developer session, which preserves execution state continuously but keeps node resources fully locked during idle periods. Under this baseline, a c3-standard-4 node accommodates only three Pods after accounting for GKE reservations.
The second strategy scales Pods down to zero between turns, releasing compute capacity while retaining files on persistent volumes. However, scaling to zero destroys active processes and memory state, forcing every subsequent prompt to initialize a fresh agent process from scratch. Consequently, platform architects need a runtime mechanism that frees node capacity while preserving the running session state.
| Architecture | Compute State | Memory Retention | 100 Agents Node Requirement | Estimated Monthly Cost |
|---|---|---|---|---|
| Dedicated Pod | Reserved continuously | Retained in memory | 34 c3-standard-4 nodes | USD 1,288 |
| Scale to Zero | Released when idle | Cleared between turns | On-demand scaling | Variable compute savings |
| Session Parking | Released to warm pool | Saved to object storage | 16 total nodes | 40% reduction from baseline |
03Agent Substrate actors and AX workspaces
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
Agent Substrate and AX divide agent operations into control plane, runtime, and sandbox layers. Agent Substrate manages the execution environment by decoupling stateful sessions from underlying machines, running each agent inside an isolated sandbox termed an actor that encapsulates processes, files, and identity. A warm pool of Kubernetes Pods hosts reusable sandboxes called workers, allowing a single worker to multiplex several actors.
Security gateways inspect inbound and outbound traffic while injecting credentials externally so that model-generated code never holds raw API keys directly. When an actor sits idle, the SuspendActor operation creates a snapshot in object storage and releases the worker Pod back to the cluster pool. The next prompt restores the actor onto any available warm worker, preserving conversational continuity across turns.
AX defines the agent identity and knowledge base through Task declarations that specify container images, commands, and limits, paired with Workspaces that define repositories and files. Executing ax suspend parks the task and preserves the /workspace directory via volume snapshots, while upcoming AX milestones will introduce resource budgets, security policies, and task audit logs.
Note: Agent Substrate 1.0 is generally available on GKE following the November 2026 donation of gVisor, Agent Substrate, and OpenRL to the Cloud Native Computing Foundation.
04Idle timeouts and worker queue limits
Cluster economics depend heavily on the specific policy chosen to trigger sandbox suspension. Because Agent Substrate issue 424 confirms that the system does not suspend idle actors automatically, the surrounding agent harness or frontend proxy must invoke the suspend API after a configured idle timeout. Until that call executes, an idle actor continues to hold its worker allocation.
If concurrent traffic exhausts available workers, incoming requests for another actor queue for a default threshold of 5 seconds before failing with an HTTP 503 error. Resource allocation also requires selecting CPU guarantees, as actors book CPU and memory limits against declared pool capacity. Setting pool capacity to actual physical node limits guarantees full resource availability without overcommitment.
05Node pool sizing and monthly cost reductions
Cost modeling for 100 coding agents at Google Cloud list prices in us-central1 demonstrates significant financial benefits from session parking. Configuring a two-minute idle timeout causes each session to occupy a worker for only 29 percent of its open duration. A warm worker pool sized for bursts at the 99th percentile plus 15 percent headroom requires 15 nodes, alongside one dedicated node running the Agent Substrate control plane and PostgreSQL database.
This architecture reduces monthly node expenses by 40 percent compared to dedicated Pods, encountering a full pool only once in roughly 3,500 turns. When prompt frequency reaches 2.5 prompts hourly, the same 15 nodes reach saturation during 3 percent of turns. At 4 prompts hourly, agents work 44 percent of the hour, expanding pool requirements to 27 nodes and reducing savings to 4 percent.
One Pod per session becomes cheaper once activity exceeds 4.5 prompts per hour, making baseline activity measurement essential prior to migration. Financial viability also requires sufficient scale, as fixed control plane overhead breaks even at roughly 37 coding sessions, or 13 sessions for chat assistants working one minute per hour. Scheduled cron jobs must scale worker pools to zero after hours to prevent idle off-shift charges.
Frequently asked questions
How does parking an AI agent session differ from scaling a Pod to zero?
Scaling to zero destroys the agent process and memory, forcing a fresh startup on every prompt. Session parking snapshots the sandbox state to object storage and restores it on a warm worker without losing context.
How much compute time does a typical coding agent spend idling?
Research on GitHub Copilot traces shows agents wait for human input 78% to 80% of an eight-hour day. A single turn averages 396.3 seconds of active compute work.
When does one Pod per session cost less than session parking?
Dedicated Pods become more cost-effective when session activity exceeds 4.5 prompts per hour, because high turn frequency keeps warm worker pools continuously occupied.
What infrastructure is required to run the Agent Substrate control plane?
The control plane requires one dedicated node running around the clock to support Agent Substrate and its PostgreSQL database, alongside a warm pool of worker Pods.
Sources
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Advertisement