Home / Blog / Agents are sessions, not requests: size GKE node pools for…
Tech News

Agents are sessions, not requests: size GKE node pools for AI agent

Google Cloud outlined a GKE architecture using Agent Substrate 1.0 to park idle AI agent sessions in object storage, cutting Kubernetes node costs by 40%.

By Dillip Chowdary • Nov 16, 2026 • Source: Google Developers Blog

Agents are sessions, not requests: size GKE node pools for AI agent
Image: Google Developers Blog

At a glance

Product
Agent Substrate 1.0 and AX
Platform
Google Kubernetes Engine
Release date
November 16, 2026
Availability
Generally available on GKE
Cost reduction
Up to 40% on Kubernetes node pools
Break-even fleet
37 coding agents or 13 chat assistants

Google Cloud released a technical framework and cost model detailing how engineering organizations can run persistent coding agent sessions on Google Kubernetes Engine without paying full compute rates for idle time, as detailed in Google Developers Blog's report. The analysis demonstrates that pairing Agent Substrate 1.0 with AX enables clusters to snapshot idle agent workspaces to object storage, cutting Kubernetes node costs by up to 40 percent.

This report examines how session-based agent workloads differ from stateless web traffic, walks through the runtime architecture of Agent Substrate and AX on GKE, and models exact cluster capacity for platform engineers. It provides node pool sizing rules, idle timeout thresholds, and traffic break-even points for teams managing coding agent fleets.

01Agent idle time between developer prompts

Unlike traditional web requests that complete within milliseconds and discard state immediately, AI coding agent sessions remain active across an entire eight-hour workday. A research study of GitHub Copilot agent traces conducted by Banruo Liu and colleagues revealed that an agent waits for developer input roughly 80 percent of the time during typical multi-turn interactions. Active work occurs only in short bursts when processing instructions.

Across those observed traces, a single turn encompassing one user prompt and the agent's complete response required an average of 396.3 seconds of compute time. Assuming a baseline developer cadence of two prompts per hour across an eight-hour shift, the agent performs active computation for only 22 percent of the day. The remaining 78 percent of cluster time is spent idling while awaiting human direction.

02Dedicated pods versus scale to zero

Engineering teams typically evaluate two conventional deployment strategies when migrating agent workloads to Kubernetes clusters. The first strategy assigns an isolated Pod running 1 vCPU and 1 GiB of memory to each developer session, which preserves execution state continuously but keeps node resources fully locked during idle periods. Under this baseline, a c3-standard-4 node accommodates only three Pods after accounting for GKE reservations.

The second strategy scales Pods down to zero between turns, releasing compute capacity while retaining files on persistent volumes. However, scaling to zero destroys active processes and memory state, forcing every subsequent prompt to initialize a fresh agent process from scratch. Consequently, platform architects need a runtime mechanism that frees node capacity while preserving the running session state.

ArchitectureCompute StateMemory Retention100 Agents Node RequirementEstimated Monthly Cost
Dedicated PodReserved continuouslyRetained in memory34 c3-standard-4 nodesUSD 1,288
Scale to ZeroReleased when idleCleared between turnsOn-demand scalingVariable compute savings
Session ParkingReleased to warm poolSaved to object storage16 total nodes40% reduction from baseline

03Agent Substrate actors and AX workspaces

Advertisement

Tech Pulse Daily

Get tomorrow's pulse first

Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.

Agent Substrate and AX divide agent operations into control plane, runtime, and sandbox layers. Agent Substrate manages the execution environment by decoupling stateful sessions from underlying machines, running each agent inside an isolated sandbox termed an actor that encapsulates processes, files, and identity. A warm pool of Kubernetes Pods hosts reusable sandboxes called workers, allowing a single worker to multiplex several actors.

Security gateways inspect inbound and outbound traffic while injecting credentials externally so that model-generated code never holds raw API keys directly. When an actor sits idle, the SuspendActor operation creates a snapshot in object storage and releases the worker Pod back to the cluster pool. The next prompt restores the actor onto any available warm worker, preserving conversational continuity across turns.

AX defines the agent identity and knowledge base through Task declarations that specify container images, commands, and limits, paired with Workspaces that define repositories and files. Executing ax suspend parks the task and preserves the /workspace directory via volume snapshots, while upcoming AX milestones will introduce resource budgets, security policies, and task audit logs.

Note: Agent Substrate 1.0 is generally available on GKE following the November 2026 donation of gVisor, Agent Substrate, and OpenRL to the Cloud Native Computing Foundation.

04Idle timeouts and worker queue limits

Cluster economics depend heavily on the specific policy chosen to trigger sandbox suspension. Because Agent Substrate issue 424 confirms that the system does not suspend idle actors automatically, the surrounding agent harness or frontend proxy must invoke the suspend API after a configured idle timeout. Until that call executes, an idle actor continues to hold its worker allocation.

If concurrent traffic exhausts available workers, incoming requests for another actor queue for a default threshold of 5 seconds before failing with an HTTP 503 error. Resource allocation also requires selecting CPU guarantees, as actors book CPU and memory limits against declared pool capacity. Setting pool capacity to actual physical node limits guarantees full resource availability without overcommitment.

05Node pool sizing and monthly cost reductions

Cost modeling for 100 coding agents at Google Cloud list prices in us-central1 demonstrates significant financial benefits from session parking. Configuring a two-minute idle timeout causes each session to occupy a worker for only 29 percent of its open duration. A warm worker pool sized for bursts at the 99th percentile plus 15 percent headroom requires 15 nodes, alongside one dedicated node running the Agent Substrate control plane and PostgreSQL database.

This architecture reduces monthly node expenses by 40 percent compared to dedicated Pods, encountering a full pool only once in roughly 3,500 turns. When prompt frequency reaches 2.5 prompts hourly, the same 15 nodes reach saturation during 3 percent of turns. At 4 prompts hourly, agents work 44 percent of the hour, expanding pool requirements to 27 nodes and reducing savings to 4 percent.

One Pod per session becomes cheaper once activity exceeds 4.5 prompts per hour, making baseline activity measurement essential prior to migration. Financial viability also requires sufficient scale, as fixed control plane overhead breaks even at roughly 37 coding sessions, or 13 sessions for chat assistants working one minute per hour. Scheduled cron jobs must scale worker pools to zero after hours to prevent idle off-shift charges.

Frequently asked questions

How does parking an AI agent session differ from scaling a Pod to zero?

Scaling to zero destroys the agent process and memory, forcing a fresh startup on every prompt. Session parking snapshots the sandbox state to object storage and restores it on a warm worker without losing context.

How much compute time does a typical coding agent spend idling?

Research on GitHub Copilot traces shows agents wait for human input 78% to 80% of an eight-hour day. A single turn averages 396.3 seconds of active compute work.

When does one Pod per session cost less than session parking?

Dedicated Pods become more cost-effective when session activity exceeds 4.5 prompts per hour, because high turn frequency keeps warm worker pools continuously occupied.

What infrastructure is required to run the Agent Substrate control plane?

The control plane requires one dedicated node running around the clock to support Agent Substrate and its PostgreSQL database, alongside a warm pool of worker Pods.

Sources

Dillip Chowdary

Author

Dillip Chowdary

Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.

Related on Tech Bytes

Advertisement

5-min tech signal

Weekday briefing for engineers who skip the noise.

No spam · Unsubscribe anytime

Advertisement

✈️ CareerPilot

Your AI job-search copilot

Match your resume against live Ashby, Greenhouse & Lever openings — fit scores, job-specific resume optimization and email alerts.

Find matching jobs →

Free Tools

Browse all tools →