New AI agent observability and governance across BigQuery and Looker

Google Cloud launched expanded AI agent observability and governance capabilities across Google Data Cloud, unifying telemetry tracking, cost management, and evaluation across BigQuery and Looker. Detailed in Google Cloud Blog's report, the update addresses operational bottlenecks that surface when moving enterprise conversational systems into production environments. The framework collects operational metrics, trace spans, and conversation logs across both first-party data agents and custom external agents, landing diagnostic telemetry directly into customer-owned BigQuery tables.

This deployment establishes visibility into agent reasoning, data access, and compute consumption for data teams, platform administrators, and FinOps engineers. It covers out-of-the-box Conversational Analytics agents within BigQuery and Looker, as well as third-party agents developed using frameworks such as Agent Development Kit (ADK) and LangGraph. The unified release balances security boundaries and compliance requirements against deep diagnostic insights, giving enterprises the tools needed to audit performance and prevent unexpected compute charges.

01Telemetry capture across first-party and custom agents

The architecture introduces a unified telemetry framework across Google Data Cloud designed to capture logs, store data in customer-managed storage, and surface metrics within existing developer tools. First-party Conversational Analytics agents emit telemetry automatically without manual configuration. Queries routed to BigQuery data agents inside Gemini Enterprise through agent-to-agent workflows, now generally available, deliver Cloud Monitoring metrics, Cloud Trace spans, and conversation logs directly into the host project.

Custom third-party agents built with ADK or LangGraph feed structured telemetry into BigQuery using single-line SDK instrumentation. The open-source ADK plugin, named BigQueryAgentAnalyticsPlugin, streams complete tool-invocation execution trees, time-series telemetry, and multimodal payloads straight into BigQuery. This setup bypasses the need for custom data pipelines, letting engineering teams capture granular agent interactions alongside standard operational metrics.

02Managed conversation logging and privacy guardrails

Telemetry lands inside a customer-owned BigQuery table called agent_events through the BigQuery Storage Write API, which is now generally available. The system separates operational metrics from conversation content to protect enterprise data privacy. Baseline Cloud Monitoring metrics and Cloud Trace spans explicitly exclude sensitive core content, leaving out user prompts, model responses, internal thoughts, and generated SQL across all organization-level dashboards.

When organizations require deep transcript auditing, agent owners can opt in to conversation logging on an individual agent basis. The managed logging pipeline inherits the dataset's IAM permissions, customer-managed encryption keys, and VPC Service Controls.

Built-in privacy controls enforce a fail-closed check before each conversational turn, automatically skipping logging whenever a referenced knowledge source relies on row-level security, column-level policy tags, or dynamic data masking. Fail-open execution ensures chat sessions continue without interruption if logging services encounter errors.

03Prebuilt dashboards and warehouse analytics

Prebuilt dashboards inside BigQuery Studio provide native monitoring for Conversational Analytics agents, tracking active sessions, total usage, latency, and primary knowledge sources. Looker (Google Cloud core) and Looker (original) provide System Activity dashboards for Conversational Analytics, monitoring user engagement and token consumption, while query reviews and user ratings in the Responses and feedback tab remain in preview. Developers can also deploy open-source Looker Blocks and Data Studio templates to visualize custom agent performance.

Because conversation logs land directly in the warehouse, engineering teams can run root-cause analysis and semantic drift detection without external software. Teams can apply BigQuery AI functions, including AI.GENERATE to summarize operational error patterns and AI.EMBED paired with VECTOR_SEARCH to cluster recurring system failures. Google also provides the open-source bigquery-agent-analytics Python SDK and Colab starter notebooks to perform LLM-as-a-judge sentiment analysis and track prompt behavior over time.

04FinOps controls and active token quotas

As conversational systems scale, granular tracking parameters allow FinOps teams to monitor compute consumption and prevent budget overruns during live operation. Each recorded conversation turn in BigQuery logs captures input tokens, output tokens, thought tokens, cached tokens, and total turn latency. Looker System Activity dashboards mirror this monitoring layer by reporting estimated input and output token consumption across enterprise business intelligence workloads.

Administrators can enforce spending boundaries using active query and token guardrails. Teams can configure per-query byte scan limits via the big_query_max_billed_bytes setting, which relies on BigQuery dry-run estimates to halt oversized queries before incurring on-demand scan charges. Google Cloud console and the gcloud CLI also support daily input and output token quotas, currently in preview, applied at the project, user, or agent level.

05Systematic agent benchmarking with Prism

Runtime observability pairs with systematic pre-deployment evaluation to verify that updates to prompts, models, or data schemas improve answer accuracy before production release. Google released Prism, an open-source evaluation tool that tests Conversational Analytics agents grounded in BigQuery tables or Looker Explores using the Conversational Analytics API. The tool validates query accuracy using reusable test suites, regex assertions, and multi-region agent discovery deployed behind Cloud IAP access layers.

Prism streams finished evaluation benchmark runs directly into day-partitioned BigQuery tables within the customer's project. This pipeline places pre-deployment benchmark scores right beside production logs stored in the agent_events table. By unifying test evaluation metrics and live runtime telemetry in a single data warehouse, platform teams can detect SQL generation regressions and measure agent reliability across model iterations.