News
Google Builds a Cloud Runtime for AI Agents
Google Cloud has announced Agent Compute Engine, a service designed to give artificial intelligence agents isolated computers for executing code, using tools and carrying out work. The infrastructure already powers Gemini agent workloads, but customer preview access is scheduled for the coming months.
The Oct. 22 announcement also expands Google's agent governance plans across cloud environments and managed devices. Google described forthcoming controls for local coding agents, cross-cloud policy enforcement, model routing and token consumption, with several gateway additions targeted for preview this quarter.
Agent Compute Engine uses small virtual machines, or microVMs, that can pause while agents wait and resume from snapshots. Copy-on-write memory sharing lets multiple agents use one operating system template, while individual Hyperdisk volumes preserve workspaces and context across sessions.
Separating Agent Decisions From Execution
Google's technical explanation separates the agent harness, which calls the model and selects actions, from the sandbox that executes commands. The harness remains outside the execution environment, with tool calls and their resulting activity correlated for auditing.
[Click on image for larger view.] Separating Agent Decisions From Sandbox Execution (source: Google).
As Google engineers Ramanathan Venkatapathy and Rohit Jnagal wrote, "We needed both the isolation of a VM and the speed and density of a container."
Each session receives a dedicated microVM. Persistent secrets, cloud service account keys and user authorization credentials stay outside the guest. A short-lived workload identity identifies sandbox traffic; the gateway checks policy and inserts backend credentials as requests leave the gateway.
Networking is also restricted: The microVM lacks a standard network interface and direct access to a virtual private cloud. Communications pass through mediation proxies. Google describes hardware-rooted protections using Titan and Titanium, encrypted snapshots and storage, and an immutable host operating system. The architecture supports interactive agents, persistent coding workspaces, evaluations and sandbox execution for other Google services.
A Kubernetes Option
Google also highlighted Agent Substrate, an open-source runtime previously announced for Google Kubernetes Engine (GKE). It runs on Kubernetes infrastructure and supports different agent frameworks, giving platform teams another execution option.
Substrate separates rapid sandbox operations from machine management. Kubernetes handles node recovery, autoscaling and worker pod lifecycles, while a dedicated execution layer manages frequent suspend-and-resume operations. Idle environments can release processor and memory resources while snapshots retain their state.
Teams can choose Cloud Hypervisor microVMs or gVisor sandboxing. An integrated gateway controls network traffic, and optional Filestore agent volumes provide shared persistent storage for collaborative workloads.
Google reports resume times below 500 milliseconds and higher workload density than traditional compute. These are vendor-reported performance claims. Its September announcement says Substrate is available to GKE customers for nonproduction workloads, with general-availability production support available through an allowlist.
Policies Follow Agent Traffic
The Agent Gateway documentation describes centralized enforcement for traffic entering and leaving agent environments. Agent Identity identifies the caller, Agent Registry lists approved destinations, and access policies determine permitted connections. Connections are blocked by default unless an explicit policy authorizes them.
[Click on image for larger view.] Agent Gateway and Agent Platform Ecosystem (source: Google).
Additional controls include Model Armor screening of prompts and tool responses and semantic governance rules expressed in plain language. The gateway generates telemetry for auditing agent interactions, including outbound calls to tools and other agents.
Availability requires care: The documentation says Agent Gateway is currently available for agents on Gemini Enterprise Agent Platform, while GKE, Cloud Run and cross-cloud support are coming through early preview. That qualification is narrower than the announcement's broad description of expanded runtime coverage. Organizations assessing deployment options should distinguish the announced scope from the availability stated in the setup documentation.
Token Budgets With Enforcement Limits
Google's forthcoming token management policies address spending at the model-request level. The documentation explains that request-count limits alone cannot adequately control consumption because individual model calls can vary substantially in token usage.
The policies track input, output and cached tokens, attribute usage to applications or agents, and enforce shared token pools. Administrators can configure hourly or daily budgets, warning thresholds and either alert-only or strict enforcement. With strict enforcement, an exhausted pool causes subsequent requests to receive an HTTP 429 response.
There is a documented limitation: Output tokens are counted after a response completes. Concurrent requests accepted before a pool is exhausted can therefore push consumption beyond its limit before later requests are blocked. Google recommends testing policies in validation mode before enabling strict enforcement.
The documentation still labels token management as coming soon, with early preview registration available and timing subject to change.
About the Author
David Ramel is an editor and writer at Converge 360.