Skip to content

fix(observability): stop using Grafana's $__ macro prefix as the SQL "All" sentinel - #5716

Merged
JSONbored merged 1 commit into
mainfrom
fix/grafana-ai-usage-all-filter-sentinel
Jul 14, 2026
Merged

fix(observability): stop using Grafana's $__ macro prefix as the SQL "All" sentinel#5716
JSONbored merged 1 commit into
mainfrom
fix/grafana-ai-usage-all-filter-sentinel

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Follow-up to fix(observability): fix Grafana dashboards showing no data after deploy #5714 (the dashboard-provisioning-orphan fix) — same live incident, but a distinct, second bug: the AI-usage dashboard's SQLite-backed panels still showed no data after fix(observability): fix Grafana dashboards showing no data after deploy #5714 resolved the provisioning crash, because the panel queries themselves were broken.
  • ai-usage.json/maintainer-reviews.json/miner-usage.json used allValue: "$__all" and compared it against a literal '$__all' to detect an unfiltered "All" selection: (${var:sqlstring} = '$__all' OR col = ${var:sqlstring}).
  • Confirmed live against the real Grafana + frser-sqlite-datasource instance: ${var:sqlstring} does not SQL-quote a value that itself starts with $__ (Grafana treats it as a macro reference, not literal data), so the substituted query carried the raw unquoted token $__all on both sides. SQLite then parsed that token as its own $__all named bind parameter, which was never supplied — every "All"-filtered panel either errored (missing named argument "__all") or silently returned zero rows, even with fresh underlying data (verified: the redacted reporting export had 61,741 rows with a last-event timestamp minutes old — this ruled out a data/pipeline problem entirely).
  • Fix: switch the sentinel to a plain string (__ALL__) that sqlstring quotes normally — verified with a direct sqlite3 execution of both the old and new pattern against the real reporting snapshot on the dedicated server (old pattern: 0 rows; new pattern: full expected row count).
  • Both dashboards' own regression tests had baked in the same wrong assumption about Grafana's substitution behavior in their simulation helpers, which is exactly why this shipped undetected — updated those too.

Scope

  • Dashboard JSON + regression tests only
  • No secrets/wallets/hotkeys/trust-scores/reward values
  • No site/, CNAME, **/lovable/**
  • No CHANGELOG.md edit

Validation

  • npx vitest run test/unit/selfhost-grafana-ai-usage-dashboard.test.ts test/unit/selfhost-grafana-miner-usage-dashboard.test.ts test/unit/ai-usage-index.test.ts — all pass, including real sqlite3-execution assertions against the fixed query text
  • Live-verified on the dedicated server: applied the identical change, restarted Grafana, confirmed correct row counts via direct sqlite3 execution against the live reporting snapshot
  • git diff --check — clean

Safety

  • No secrets in code/comments/tests
  • Already verified safe and working in production before opening this PR

…"All" sentinel

The ai-usage/maintainer-reviews/miner-usage dashboards used allValue: "$__all"
and compared it against a literal '$__all' to detect an unfiltered "All"
selection. Confirmed live against a real Grafana + frser-sqlite-datasource
instance: ${var:sqlstring} does not sql-quote a value that itself starts with
"$__" (Grafana treats it as a macro reference, not literal data), so the
substituted query carried the raw unquoted token $__all on both sides of the
comparison. SQLite then parsed that token as its own $__all named bind
parameter, which was never supplied -- every "All"-filtered panel query
either errored ("missing named argument \"__all\"") or silently returned zero
rows, even though the underlying reporting data was present and fresh.

Switches the sentinel to a plain string (__ALL__) that sqlstring quotes
normally, and updates both dashboards' own regression test simulations (which
had baked in the same wrong assumption about Grafana's substitution behavior,
which is why this shipped undetected).
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 14, 2026
@JSONbored
JSONbored merged commit 5f187c2 into main Jul 14, 2026
13 of 14 checks passed
@JSONbored
JSONbored deleted the fix/grafana-ai-usage-all-filter-sentinel branch July 14, 2026 07:06
@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

❌ 7 Tests Failed:

Tests completed Failed Passed Skipped
16021 7 16014 12
View the top 3 failed test(s) by shortest run time
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes the issue-activity panels to the selected $repo, same as the PR panels
Stack Traces | 0.00515s run time
AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:361:44
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes every review_targets panel query to the selected repo with Grafana SQL-string escaping
Stack Traces | 0.00571s run time
AssertionError: expected 'SELECT count(*) AS prs FROM review_ta…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS prs FROM review_targets WHERE (submitter NOT LIKE '%[bot]%' OR submitter IS NULL) AND (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(updated_at) >= ${__from:date:seconds} AND unixepoch(updated_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:384:32
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > adds local, webhook-observed issue-activity stat panels alongside the review_targets PR panels (#3716, switched off the GitHub API 2026-07)
Stack Traces | 0.016s run time
AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:343:33
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > filters the pull request table to the selected time window
Stack Traces | 0.0227s run time
AssertionError: expected '' to contain 'owner/repo|2|new|commented|comment|ne…'

- Expected
+ Received

- owner/repo|2|new|commented|comment|new row|2026-06-29T21:00:00Z

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:446:18
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > actually narrows the PRs-tracked count to a selected $repo, and 'All' still includes every repo
Stack Traces | 0.0254s run time
AssertionError: expected '0' to be '3' // Object.is equality

Expected: "3"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:478:22
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > drops a bot-authored release PR from the tracked-PR count and table (#4685-follow-up)
Stack Traces | 0.0674s run time
AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:546:26
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > issue-activity panels count real rows correctly by state and window
Stack Traces | 0.386s run time
AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:416:20

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 14, 2026
JSONbored added a commit that referenced this pull request Jul 14, 2026
…aintainer-reviews dashboard test (#5719)

#5716 moved every panel in grafana/dashboards/maintainer-reviews.json from
the "\$__all" Grafana-macro-prefixed sentinel to a plain "__ALL__" literal,
but missed updating this test file's assertions and its expandGrafanaRange()
helper default. The stale "\$__all" default meant simulating the "All repos"
selection built a WHERE clause that could never match the dashboard's real
"__ALL__" sentinel, so every issue-activity/PR-panel row-count assertion
silently fell through to the single-repo branch and returned 0 rows instead
of the true total.
JSONbored added a commit that referenced this pull request Jul 14, 2026
Three independent, pre-existing gate breaks, none introduced by this branch --
found while chasing why release-please's PRs (#5710-5713) looked flaky under
local parallel test runs, and confirmed to reproduce identically on a clean
origin/main checkout:

- test/unit/agent-sdk-driver.test.ts, ai-summaries.test.ts, upstream-ruleset.test.ts:
  each does real, non-trivial work (git subprocess round-trips, a 42-entry
  forbidden-word sweep, 29 sequential D1-backed scenarios) that reliably exceeds
  vitest's 15s default under load. Widened each to an explicit, realistic timeout
  instead of leaving them to flake.
- test/unit/selfhost-grafana-dashboard.test.ts: 5f187c2 (#5716) fixed a real
  production bug -- Grafana's ${var:sqlstring} doesn't sql-quote a value that
  starts with $__, so the old $__all "All repos" sentinel leaked through
  unescaped and SQLite misparsed it as its own bind parameter, silently zeroing
  every "All"-filtered panel. That commit updated the dashboard JSON and two
  sibling test files but missed this one.
- apps/gittensory-ui/public/openapi.json + 3 route/component .tsx files:
  40e6cdf (#5715, the @gittensory -> @Loopover bot-mention rename) changed
  copy text but never regenerated the OpenAPI spec or ran prettier on the
  reflowed JSX text nodes, so ui:openapi:check and ui:lint both fail on main.

Regenerated the spec (npm run ui:openapi) and ran eslint --fix on the affected
files; both are mechanical, zero-semantic-diff outputs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant