Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion apps/gittensory-ui/src/routes/docs.self-hosting-operations.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,22 @@ review_context_fetch_failed`}
The observability profile starts Prometheus, Alertmanager, Loki, Promtail, and Grafana with
dashboards for infra, review activity, and AI usage.
</p>
<p>
Postgres installs also expose database internals through the bundled Postgres exporter:
connection pressure, lock waits, long transactions, deadlocks, database/table growth, dead
tuples, autovacuum activity, and backup freshness. Backup freshness appears when the{" "}
<code>backup</code> profile is active.
</p>
<p>
When OpenTelemetry and Sentry are enabled, job audit logs and Sentry events include
trace_id/span_id fields so an operator can jump from a failed job or issue to the matching
trace in Grafana or Tempo.
</p>
<CodeBlock lang="bash" code={`docker compose --profile observability up -d`} />
<CodeBlock
lang="bash"
code={`docker compose --profile postgres --profile observability up -d
docker compose --profile postgres --profile observability --profile backup up -d`}
/>

<h2>Sentry tracing</h2>
<p>
Expand Down Expand Up @@ -129,6 +139,10 @@ review_context_fetch_failed`}
<li>Webhook deliveries are recent and have 2xx responses.</li>
<li>AI usage matches expected review volume and model/effort choices.</li>
<li>REES and RAG failures are visible and bounded.</li>
<li>
Postgres connections, lock waits, slow transactions, dead tuples, and table growth are
stable.
</li>
<li>Backups are recent and restore-tested.</li>
</ul>

Expand Down
36 changes: 36 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,22 @@ services:
DEFAULT_POOL_SIZE: "20"
AUTH_TYPE: md5

# Postgres internals exporter. Starts with the Postgres profiles so Prometheus can scrape it when
# observability is enabled, without starting against a missing DB on SQLite-only installs.
postgres-exporter:
image: quay.io/prometheuscommunity/postgres-exporter:v0.20.0
restart: unless-stopped
profiles: ["postgres", "pgbouncer"]
depends_on:
postgres:
condition: service_healthy
environment:
DATA_SOURCE_URI: "postgres:5432/gittensory?sslmode=disable"
DATA_SOURCE_USER: gittensory
DATA_SOURCE_PASS: ${POSTGRES_PASSWORD:-CHANGEME}
expose:
- "9187"

# ── Qdrant (--profile qdrant) ─────────────────────────────────────────────
# Dedicated vector database for RAG — replaces the built-in sqlite-vec / pgvector when
# QDRANT_URL=http://qdrant:6333 is set. Scales to millions of vectors with ANN search.
Expand Down Expand Up @@ -528,6 +544,26 @@ services:
command:
- "apk add --no-cache sqlite postgresql16-client curl >/dev/null 2>&1 && while true; do sh /backup.sh || echo '[backup] run failed'; sleep ${BACKUP_INTERVAL_SECONDS:-86400}; done"

# Read-only backup freshness exporter. It serves file-age metrics from the retained backup volume so
# Prometheus can alert on missing or stale database backups without Grafana reading the live app DB.
backup-exporter:
image: alpine:3.20
restart: unless-stopped
profiles: ["backup"]
volumes:
- gittensory-backups:/backups:ro
- ./scripts/backup-metrics.sh:/backup-metrics.sh:ro
expose:
- "9101"
command:
- /bin/sh
- -c
- "sh /backup-metrics.sh"
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:9101/metrics | grep -q '^gittensory_backup_latest_timestamp_seconds'"]
interval: 30s
retries: 5

volumes:
gittensory-data:
gittensory-pg:
Expand Down
Loading
Loading