test(observability): guard against today's two Grafana bug classes

[JSONbored]

Summary

Follow-up to #5714/#5716 — turns today's live-incident lessons into permanent CI guards so they can't recur silently.

• selfhost-grafana-no-dollar-underscore-sentinel.test.ts: fails CI if any dashboard uses a Grafana-reserved $__-prefixed value as a hand-rolled SQL "All" sentinel — the exact bug fixed in fix(observability): stop using Grafana's $__ macro prefix as the SQL "All" sentinel #5716. Confirmed live that ${var:sqlstring} leaves such values unquoted, which SQLite then misparses as its own bind parameter. Verified this test actually catches the original broken pattern (temporarily reintroduced it, confirmed the test fails, reverted).
• selfhost-grafana-datasource-uids.test.ts: pins every provisioned datasource uid and asserts every dashboard panel/template-variable references one that actually exists in a provisioning file. A silent datasource-uid rename without a live-instance migration crash-loops the entire Grafana container on next recreate (confirmed live, separately from fix(observability): fix Grafana dashboards showing no data after deploy #5714 — a distinct incident during today's investigation, fixed with a one-time production DB correction, not a code change since it isn't reproducible from a fresh install). This test can't catch a missing migration step by itself, but it forces a deliberate test update on any future rename, which is the intended prompt to also plan that step.
• Surfaces one pre-existing, accepted gap along the way: github-prs.json's "github" datasource has no git-tracked provisioning source at all — the grafana-github-datasource plugin needs a live PAT, which must never be committed, so it's configured manually per self-host instance. Explicitly allowlisted with a comment, not silently ignored.

Scope

• Test-only change
• No secrets/wallets/hotkeys/trust-scores/reward values
• No site/, CNAME, **/lovable/**
• No CHANGELOG.md edit

Validation

• npx vitest run test/unit/selfhost-grafana-no-dollar-underscore-sentinel.test.ts test/unit/selfhost-grafana-datasource-uids.test.ts — all pass
• git diff --check — clean

Safety

• No secrets in code/comments/tests
• Test-only, zero runtime behavior change

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

❌ 7 Tests Failed:

Tests completed	Failed	Passed	Skipped
16047	7	16040	12

View the full list of 7 ❄️ flaky test(s)

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > actually narrows the PRs-tracked count to a selected $repo, and 'All' still includes every repo

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0427s run time

AssertionError: expected '0' to be '3' // Object.is equality

Expected: "3"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:478:22

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > adds local, webhook-observed issue-activity stat panels alongside the review_targets PR panels (#3716, switched off the GitHub API 2026-07)

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0202s run time

AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:343:33

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > drops a bot-authored release PR from the tracked-PR count and table (#4685-follow-up)

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0457s run time

AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:546:26

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > filters the pull request table to the selected time window

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0257s run time

AssertionError: expected '' to contain 'owner/repo|2|new|commented|comment|ne…'

- Expected
+ Received

- owner/repo|2|new|commented|comment|new row|2026-06-29T21:00:00Z

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:446:18

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > issue-activity panels count real rows correctly by state and window

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0453s run time

AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:416:20

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes every review_targets panel query to the selected repo with Grafana SQL-string escaping

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.00742s run time

AssertionError: expected 'SELECT count(*) AS prs FROM review_ta…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS prs FROM review_targets WHERE (submitter NOT LIKE '%[bot]%' OR submitter IS NULL) AND (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(updated_at) >= ${__from:date:seconds} AND unixepoch(updated_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:384:32

test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes the issue-activity panels to the selected $repo, same as the PR panels

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.00598s run time

AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:361:44

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}