Skip to content

chore: Metric Views typegen#433

Draft
atilafassina wants to merge 7 commits into
mainfrom
mv-typegen
Draft

chore: Metric Views typegen#433
atilafassina wants to merge 7 commits into
mainfrom
mv-typegen

Conversation

@atilafassina

@atilafassina atilafassina commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Build-time type generation for UC Metric Views.

When config/queries/metric-views.json is present, the type generator runs DESCRIBE TABLE EXTENDED ... AS JSON per declared metric view and emits two artifacts:

  1. shared/appkit-types/metric.d.ts — the MetricRegistry module augmentation. Each entry carries typed measures/dimensions row fields plus measureKeys/dimensionKeys/timeGrains literal unions, the base the MeasureKey<K>/DimensionKey<K>/MetricRow<K>/TimeGrain<K> helpers derive from on the appkit-ui side.

  2. shared/appkit-types/metrics.metadata.json — the semantic-metadata bundle, entries shaped { measures, dimensions } (display names, format specs, descriptions, time-grain hints). Frontend-safe by construction: UC FQNs and execution lanes are deliberately excluded.

Note

Dormancy invariant: absent config means nothing executes — zero artifacts, zero logs, no fallback to any legacy filename. Apps that never adopt metric views see no change, which is what keeps merging this incrementally safe ahead of the runtime.

Config contract (already merged)

config/queries/metric-views.json, entity-first metricViews map per the #429 metric-source schema:

{
  "metricViews": {
    "revenue": { "source": "main.finance.revenue_metrics" },
    "customer_metrics": { "source": "main.cs.customer_metrics", "executor": "user" }
  }
}

executor is app_service_principal (default) or user (per-user OBO); the internal sp/obo lane is derived at the parse boundary, so downstream code only ever sees lanes.

Failure semantics

  • Config absent → fully dormant.
  • Broken config (unparseable JSON, unknown fields, invalid keys/FQNs/executors) → throws loudly at build time.
  • Per-key DESCRIBE failures (typo'd view, permissions) → warn-and-continue, shipping empty measure/dimension allowlists for that key. This arms the runtime's fail-closed gate (next PR in the chain): an empty allowlist means the route 503s rather than passing unvalidated identifiers through.

Vite plugin grows metricOutFile/metricMetadataOutFile options, and the dev watcher regenerates on metric-views.json edits through the exact same single-flight regen flow as .sql files.

SQL feature parity (post-review)

Review of the metric path surfaced three gaps versus the query path's #406 warehouse state machine: (1) a cold/starting warehouse was misdiagnosed as per-view failure (baking empty never allowlists into .d.ts), (2) no cache — every pass re-described every view, and (3) DESCRIBEs ran sequentially. Parity with the query path's state machine is the shipping bar; four commits close it:

  • 35d1800 — degraded-vs-failure classification matrix + permissive degraded .d.ts rendering. Non-terminal DESCRIBE outcomes (warehouse not ready, transient errors) classify as degraded: measureKeys/dimensionKeys/timeGrains open to string and row fields to Record<string, unknown> so app code keeps compiling until a successful pass refreshes them. Accurate never unions are reserved for confirmed-empty views; bundle shape is unchanged ({ measures: {}, dimensions: {} }).
  • 5cee6ec — single WorkspaceClient per generation pass (status probe, preflight, and default DESCRIBE fetcher share one lazy instance; warm/no-op passes construct zero clients) + blocking-mode metric preflight mirroring the query flow (probe → decide → wait / start+wait). Deliberately split from the query preflight — queries and metric views may bind to different warehouses in the future. Fatal-on-DELETED rides the same TypegenFatalError path.
  • f02fe5a — bounded-concurrency DESCRIBEs: chunks of 10 via Promise.allSettled, order-preserving results, per-key failure isolation unchanged.
  • 3f426e7metrics cache section in .appkit-types-cache.json (no cache version bump — query entries untouched). Entries hash as md5(source|lane); degraded results persist retry: true and converge via the existing warehouse watch; last-known-good schemas are served while the warehouse is down; noCache is now meaningful for metrics; fully-warm passes make zero warehouse calls.

Suite is now 2900 tests; 98 new metric-path test blocks across the three touched test files.

This pull request and its description were written by Isaac.

build-time type generator for UC Metric Views — reads config/queries/metric-views.json
(entity-first metricViews map per the #429 schema; executor app_service_principal|user ->
internal sp/obo lane), runs DESCRIBE TABLE EXTENDED ... AS JSON per declared view, and emits
two artifacts: the MetricRegistry .d.ts augmentation (metric.d.ts:
MeasureKey/DimensionKey/MetricRow/TimeGrain) and the metrics.metadata.json semantic-metadata
bundle (entries { measures, dimensions }, frontend-safe). Absent config = fully dormant
(zero artifacts, zero logs). Per-key DESCRIBE failures warn-and-continue and ship empty
allowlists (arms the runtime fail-closed gate, next PR in chain); broken config throws
loudly. Vite plugin: metricOutFile/metricMetadataOutFile options + watcher regen on
metric-views.json edits. Re-implemented from #341 (a7bd698) on current main (post-#406
non-blocking preflight surface); 3rd PR of the #341 decomposition after #427/#429.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds build-time type generation for Unity Catalog Metric Views, emitting a MetricRegistry module augmentation plus a frontend-safe semantic metadata bundle when config/queries/metric-views.json is present.

Changes:

  • Introduces metric-view config parsing/validation + DESCRIBE-driven schema extraction, emitting metric.d.ts and metrics.metadata.json.
  • Extends the Vite typegen plugin with metric output options and watcher support for metric-views.json changes.
  • Adds extensive unit + snapshot coverage for metric registry generation and plugin option plumbing.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/appkit/src/type-generator/vite-plugin.ts Adds metric output options and triggers regeneration on metric-views.json edits.
packages/appkit/src/type-generator/index.ts Wires metric-view generation into generateFromEntryPoint and exports metric artifact constants/types.
packages/appkit/src/type-generator/metric-registry.ts Implements metric config resolution, DESCRIBE parsing, type/metadata emission, and sync failure reporting.
packages/appkit/src/type-generator/tests/vite-plugin.test.ts Tests watcher behavior for metric-views.json and option plumbing for metric outputs.
packages/appkit/src/type-generator/tests/index.test.ts Tests end-to-end emission/dormancy behavior for metric artifacts in generateFromEntryPoint.
packages/appkit/src/type-generator/tests/metric-registry.test.ts Adds comprehensive unit tests for config validation, extraction, time grains, and metadata formatting.
packages/appkit/src/type-generator/tests/snapshots/metric-registry.test.ts.snap Snapshot coverage for emitted metric.d.ts and metrics.metadata.json.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +275 to +281
const resolution = resolveMetricConfig(metricConfig);
const fetcher =
metricFetcher ?? createWorkspaceDescribeFetcher(warehouseId);
const { schemas: metricSchemas, failures } = await syncMetrics(
resolution,
fetcher,
);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55d4f9. The metric block now gates on the #406 preflight contract: in non-blocking mode it makes a single read-only warehouses.get status probe (verified it can never start a warehouse — startWarehouse is a separate, uninvoked path) and only runs DESCRIBEs when the warehouse is already RUNNING. When it's not running (or the probe itself fails), all DESCRIBEs are skipped — createWorkspaceDescribeFetcher is never constructed — and both artifacts are still emitted in degraded form (every configured key with empty measures/dimensions), which the Vite plugin's warehouse-watch regen (it re-invokes generation in blocking mode) refreshes once the warehouse comes up. This mirrors the query path's "types are always written, degraded refreshes later" behavior. blocking mode and an explicitly injected metricFetcher (test seam, no warehouse involved) bypass the gate. Covered by 5 new tests: skip-when-stopped (fetcher-never-called + degraded artifacts + single info line), non-blocking+RUNNING, blocking-no-probe, injected-fetcher bypass, and probe-failure degrade.

Comment thread packages/appkit/src/type-generator/metric-registry.ts Outdated
Comment thread packages/appkit/src/type-generator/metric-registry.ts Outdated
Comment thread packages/appkit/src/type-generator/metric-registry.ts Outdated
@atilafassina atilafassina marked this pull request as draft June 10, 2026 18:31
@atilafassina atilafassina changed the title feat(appkit): metric-registry type generation chore: Metric Views typegen Jun 10, 2026
…c failure logs

Copilot review response (#433): (1) the metric emit block now gates DESCRIBEs
on warehouse state in non-blocking mode — one read-only status GET (never
starts a warehouse); when not RUNNING (or the probe fails) it skips all
DESCRIBEs and emits degraded artifacts (every configured key with empty
measures/dimensions) that the vite plugin's warehouse-watch regen (blocking
mode) refreshes once the warehouse is up. Blocking mode and injected
metricFetcher bypass the gate. (2) syncMetrics is now log-free: the three
internal per-failure warns are removed and the generateFromEntryPoint caller
owns surfacing failures exactly once.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
…aded metric types permissively

Signed-off-by: Atila Fassina <atila@fassina.eu>
…preflight

Signed-off-by: Atila Fassina <atila@fassina.eu>
Signed-off-by: Atila Fassina <atila@fassina.eu>
…d last-known-good degradation

Signed-off-by: Atila Fassina <atila@fassina.eu>
Signed-off-by: Atila Fassina <atila@fassina.eu>
@atilafassina

Copy link
Copy Markdown
Contributor Author

Pushed 4 new commits adding SQL-state-machine parity for the metric path (plus one small docs commit, 6d0953a, documenting the convergence contract on the typegen options).

The metric DESCRIBE path now behaves like the query path's #406 warehouse state machine: a cold or starting warehouse is classified as degraded (permissive types now, retry-flagged cache entries that converge on the next describe-capable pass — the dev warehouse watch triggers that automatically) instead of being misreported as per-view failure; the whole pass shares a single lazily-constructed WorkspaceClient; blocking mode gets a real preflight; DESCRIBEs run with bounded concurrency; and schemas are cached so warm passes make zero warehouse calls and last-known-good types survive a down warehouse.

  • 35d1800 — classify non-terminal DESCRIBE outcomes as degraded; render degraded entries permissively (string key unions, Record<string, unknown> rows). Exact never unions remain only for confirmed-empty views; metadata bundle shape unchanged.
  • 5cee6ec — single WorkspaceClient per generation pass + blocking-mode metric preflight (probe → decide → wait/start+wait; only DELETED/DELETING is fatal, via the same TypegenFatalError path). Kept separate from the query preflight since the two paths may bind different warehouses later.
  • f02fe5a — bounded-concurrency DESCRIBEs (chunks of 10 via Promise.allSettled, order-preserving, per-key isolation unchanged).
  • 3f426e7metrics section in .appkit-types-cache.json (md5(source|lane) hashing, retry-driven convergence, last-known-good degradation, noCache now applies to metrics; no cache version bump — query entries untouched).

This resolves the three review findings: (1) cold-warehouse misdiagnosed as failure, (2) no metric cache, (3) sequential DESCRIBEs.

This pull request and its description were written by Isaac.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants