Skip to content

feat(spanner): add optional endpoint attribute to client metrics for location-aware routing - #13740

Draft
rahul2393 wants to merge 1 commit into
googleapis:mainfrom
rahul2393:feat/spanner-endpoint-attribute
Draft

feat(spanner): add optional endpoint attribute to client metrics for location-aware routing#13740
rahul2393 wants to merge 1 commit into
googleapis:mainfrom
rahul2393:feat/spanner-endpoint-attribute

Conversation

@rahul2393

@rahul2393 rahul2393 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an optional endpoint attribute to Spanner client metrics attempt-level instruments. When Spanner Omni location-aware routing selects a specific endpoint for a request, that endpoint is recorded as a metric attribute, so operators can attribute client-side attempt latency and attempt counts to the routed endpoint.

Scope

Stacked on top of #13741 (Client Metrics export). This PR adds only the endpoint attribute:

  • ENDPOINT_KEY attribute plumbed through the custom-export views — attempt latency and attempt count only. Operation-level views and the Cloud Monitoring views deliberately drop it.
  • KeyAwareChannel records the routed endpoint on the per-attempt tracer at endpoint-selection time.
  • Tests covering endpoint presence on attempt metrics and its absence on operation, Cloud Monitoring, and default-client paths.

Stacking note

Until #13741 merges, this PR's diff also includes #13741's commit. Once #13741 lands, this branch will be rebased onto main for a clean endpoint-only diff. Do not merge before #13741.

@google-cla

google-cla Bot commented Jul 14, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for exporting Cloud Spanner client-side metrics to a caller-owned OpenTelemetry destination. It adds the MetricsProvider interface and its implementations (DefaultMetricsProvider, NoopMetricsProvider, and CustomOpenTelemetryMetricsProvider), allowing custom metrics pipelines to be configured independently of the default Cloud Monitoring export. Additionally, it integrates endpoint routing labels for Spanner Omni and includes comprehensive test coverage. The reviewer feedback suggests a performance optimization in BuiltInMetricsConstant.java to pre-compute the common attribute keys as a static final field rather than streaming and mapping them on every call to baseAttributesFilter.

Comment on lines +298 to 307
private static Set<String> baseAttributesFilter(boolean includeEndpointAttribute) {
Set<String> attributesFilter =
BuiltInMetricsConstant.COMMON_ATTRIBUTES.stream()
.map(AttributeKey::getKey)
.collect(Collectors.toCollection(HashSet::new));
if (includeEndpointAttribute) {
attributesFilter.add(ENDPOINT_KEY.getKey());
}
return attributesFilter;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The set of common attribute keys is derived from COMMON_ATTRIBUTES which is a static constant. Instead of streaming, mapping, and collecting COMMON_ATTRIBUTES on every single call to baseAttributesFilter, we can pre-compute this set of keys once as a static final field and reuse it. This improves both readability and performance during initialization.

  private static final Set<String> COMMON_ATTRIBUTE_KEYS =
      BuiltInMetricsConstant.COMMON_ATTRIBUTES.stream()
          .map(AttributeKey::getKey)
          .collect(Collectors.toSet());

  private static Set<String> baseAttributesFilter(boolean includeEndpointAttribute) {
    Set<String> attributesFilter = new HashSet<>(COMMON_ATTRIBUTE_KEYS);
    if (includeEndpointAttribute) {
      attributesFilter.add(ENDPOINT_KEY.getKey());
    }
    return attributesFilter;
  }

…location-aware routing

Adds the ENDPOINT_KEY metric attribute: KeyAwareChannel labels each
attempt with the endpoint the location-aware fastpath routed it to, and
the custom-export attempt views allow the label through their attribute
filter. Cloud Monitoring views never include it.

Stacked follow-up to the client metrics export change (googleapis#13679); until
that merges this commit also carries the client metrics export work.
Merge after googleapis#13679 lands.
@rahul2393
rahul2393 force-pushed the feat/spanner-endpoint-attribute branch from a8b2e77 to 307e31a Compare July 14, 2026 06:06
rahul2393 added a commit that referenced this pull request Jul 30, 2026
… export (#13741)

## Summary
Adds **Client Metrics**: a new, opt-in feature that exports Spanner's
client-side metrics to a caller-provided OpenTelemetry pipeline (OTLP,
Prometheus, any exporter), including on Spanner Omni where the existing
Cloud Monitoring metrics are unavailable.

Fully opt-in and fully decoupled from the existing built-in (Cloud
Monitoring) metrics. Default behavior is unchanged: without a
client-metrics provider, nothing new happens, and built-in metrics
continue to export to Cloud Monitoring exactly as before.

> **Scope note:** This change was split out of a larger PR into two
focused PRs. This PR contains **only** the Client Metrics export
feature. The optional `endpoint` metric attribute for location-aware
routing is a separate follow-up: #13740.

## Two independent features
- **Built-in metrics** (existing, unchanged): export to Google Cloud
Monitoring. Controlled by `setBuiltInMetricsEnabled` and the
`SPANNER_DISABLE_BUILTIN_METRICS` environment variable, exactly as
today. Not available on Spanner Omni.
- **Client metrics** (new): export the client instruments to a
caller-owned OpenTelemetry. Controlled **solely** by
`setClientMetricsProvider(...)` — a `CustomOpenTelemetryMetricsProvider`
turns it on; `NoopMetricsProvider` (or no provider) turns it off. Works
on all instance types, Omni included.

The two are decoupled: `setBuiltInMetricsEnabled` and the env var affect
only the built-in Cloud Monitoring sink and have no effect on client
metrics; the client-metrics provider affects only the caller-owned sink
and has no effect on built-in metrics. Under the hood these are the same
client instruments, exported under the distinct `spanner/client`
namespace — the difference is the export path, not the metrics.

## Emulator handling
Client metrics are not recorded when Spanner is pointed at the emulator.
To make that reliable when the emulator is configured programmatically
(via `setEmulatorHost(...)`) and not only through the
`SPANNER_EMULATOR_HOST` environment variable, this PR broadens emulator
detection: `SpannerOptions.isEmulatorEnabled()` now also recognizes the
builder-configured emulator host, and the connection-check error message
is generalized to describe both the environment-variable and
programmatic configuration paths. This gates both the client and Cloud
Monitoring metrics sinks off against the emulator using a single
detection predicate. Called out explicitly here for reviewer visibility.

## Motivation
Client metrics currently export only to Cloud Monitoring, which is
unavailable on Spanner Omni. Customers running on Omni (or who
standardize on their own observability stack) had no way to receive
these metrics. This lets them route the metrics to any OpenTelemetry
exporter, independently of the built-in Cloud Monitoring configuration.

## API
```java
SdkMeterProviderBuilder meterProviderBuilder = SdkMeterProvider.builder();
SpannerMetrics.configureMeterProviderBuilder(meterProviderBuilder);
// ... attach your exporter to meterProviderBuilder ...
OpenTelemetry otel =
    OpenTelemetrySdk.builder().setMeterProvider(meterProviderBuilder.build()).build();

SpannerOptions options =
    SpannerOptions.newBuilder()
        .setClientMetricsProvider(new CustomOpenTelemetryMetricsProvider(otel))
        .build();
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant