[lvaroqui]

Export the traces and logs that divan benchmarks emit with tracing and log, for the telemetry mode of the CodSpeed runner (CodSpeedHQ/codspeed#589) to collect. Stacked on #201, whose round markers tell which benchmark each span and log belongs to.

Using it takes the telemetry feature of codspeed-divan-compat and a telemetry build:

cargo codspeed build -m telemetry            # or with other modes, e.g. -m simulation,telemetry
codspeed run -m telemetry -- cargo codspeed run

When CODSPEED_OTEL_ENDPOINT is set, divan::main() installs a tracing subscriber exporting over OTLP/HTTP:

• spans as traces;
• tracing events and log records as logs, carrying the trace and span IDs of the span they are emitted in.

CODSPEED_TELEMETRY_FILTER takes EnvFilter directives, info by default. Benchmarks that install their own subscriber add divan::telemetry::layer() to it.

cargo-codspeed

• A new telemetry measurement mode builds the benchmarks to run once, like simulation, with tracing kept. It can be built along other modes.
• Every other mode compiles tracing out (max_level_off, behind --cfg=codspeed_tracing_off), so spans and events never weigh on the measurements. --keep-tracing opts out, e.g. to benchmark a subscriber.
• The OpenTelemetry dependencies sit in a cfg(codspeed_telemetry) target table, a cfg only the telemetry build sets. With the feature on, measured builds compile 31 crates of codspeed instead of 157, and a project without the feature gets nothing new in its lockfile.

Timing

Spans and log records keep the wall clock times of the SDK. With instrument hooks, they also carry their times on the monotonic clock of the benchmark markers, which wall clock adjustments don't move: spans as codspeed.monotonic_start_ns and codspeed.monotonic_end_ns, log records as codspeed.monotonic_time_ns.

Worth a careful look

• How the span times are taken, in crates/codspeed/src/telemetry.rs. tracing-opentelemetry takes a span's start when the tracing span is created, but only starts the OpenTelemetry span when it is first entered, e.g. on the first poll of an instrumented future. A layer after it starts the span on creation, through get_otel_context, to set the start then. Its end is taken when the span is last exited, and the SDK only appends attributes, so a layer before it records the last exit and sets the end once, when the span closes.
• The filters hint TRACE as their max level, so that LogTracer keeps the level of the EnvFilter: without it, filter_fn lifted it to TRACE.
• init warms the subscriber up with a span and an event that are not exported: the first span of the process took 18 to 31 µs longer than the next ones.
• The exporter's own HTTP stack (hyper, reqwest, opentelemetry_*, …) is always filtered out, or exporting would emit more spans in an endless loop.
• reqwest pulls icu_* 2.x crates, which require rustc 1.88: they are resolved with the MSRV-aware fallback resolver. sharded-slab fails to build with the lazy_static 1.0 that minimal-versions picks, hence a lazy_static = "1.1" floor.

crates/divan_compat/examples/benches/telemetry.rs emits telemetry in each of these ways: nested spans with events, an event outside of any span, records of the log crate, and nothing at all. The telemetry tests only build with --cfg codspeed_telemetry; CI runs them, and checks the MSRV and minimal versions, with it.

Closes COD-3862

[codspeed]

Merging this PR will regress 18 benchmarks

⚠️ 61 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

• tsc (aarch64)
• small_drop
• "*benchmark/" '
• iter
• iter_with_large_setup
• iter_batched_ref_large_input
• iter_with_large_drop
• iter_batched_large_input
• iter_batched_small_input
• iter_batched_ref_small_input
• …and 51 more

⚠️ 9 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• iter_manual_simple
• recursive_memoized[HashMap<u64, u64>, 0]
• print_env_hello
• fibo_recursive_cached
• allocate
• bench_large_input_local
• bench_large_input
• system_time
• instant

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 35 improved benchmarks
❌ 18 regressed benchmarks
✅ 326 untouched benchmarks
⏩ 215 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Walltime	generate_combinations[9]	5.7 µs	8.1 µs	-29.93%
❌	Walltime	bench_array2[10]	5 ns	7 ns	-28.57%
❌	Walltime	bench_array1[10]	5 ns	7 ns	-28.57%
❌	Walltime	div	3 ns	4 ns	-25%
❌	CPU Simulation	recursive_memoized[BTreeMap<u64, u64>, 0]	717.2 ns	828.3 ns	-13.41%
❌	Walltime	init_array[1000]	1.5 µs	1.7 µs	-10.46%
❌	Walltime	permutations[6]	68.5 µs	76.4 µs	-10.38%
❌	CPU Simulation	a	568.5 ns	624.1 ns	-8.9%
❌	CPU Simulation	a	568.5 ns	624.1 ns	-8.9%
❌ ⚠️	CPU Simulation	iter_manual_simple	839.4 ns	894.9 ns	-6.21%
❌	CPU Simulation	process_items_with_counter	1.9 µs	2 µs	-5.52%
❌	Walltime	bench_array2[42]	35 ns	37 ns	-5.41%
❌	Walltime	bench_array1[42]	35 ns	37 ns	-5.41%
❌	Walltime	init_array[42]	35 ns	37 ns	-5.41%
❌	Walltime	Iterative[21]	21 ns	22 ns	-4.55%
❌	Walltime	from_elem_decimal[1024]	213 ns	223 ns	-4.48%
❌	Walltime	slice_into_vec_with_bytes	53 ns	55 ns	-3.64%
❌	CPU Simulation	string_copy_with_bytes_counter	1.6 µs	1.7 µs	-3.22%
⚡	Walltime	iterative[0]	4 ns	1 ns	×4
⚡	Walltime	print_env_hello	4.9 µs	1.9 µs	×2.5
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-3862-featrust-otel-cargo-feature-in-codspeed-divan-compat (5468bd4) with main (76685ee)}

Footnotes

1. 215 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[codspeed]

Unable to generate the performance report

The run did not report any results. Please check the documentation to check your benchmarking setup.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; neither previous finding remains outstanding.

Summary

Adds OTLP export for Divan benchmark spans and logs, with a separate telemetry build mode.

• Divan benchmarks send their traces and logs to OTLP.
• Cargo CodSpeed runs benchmarks once in telemetry mode.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Build["cargo codspeed build -m telemetry"] --> Bench["Divan benchmark"]
    Bench --> Spans["tracing spans"]
    Bench --> Logs["tracing events and log records"]
    Spans --> Traces["OTLP traces"]
    Logs --> Records["OTLP logs"]
    Traces --> Receiver["CODSPEED_OTEL_ENDPOINT"]
    Records --> Receiver

Loading

_{Reviews (2) · Last reviewed commit: "perf(codspeed): warm the telemetry up be..." · Reviewed by Greptile}