From 4143e814971ecae2b936bd3454e5b4416bcf1953 Mon Sep 17 00:00:00 2001 From: Tommaso Morganti Date: Tue, 10 Nov 2026 12:34:55 +0100 Subject: [PATCH] docs(infrastructure): document the SSO gateway and how to protect an app with it --- .../Guides/deploy-a-new-app.mdx | 3 + content/docs/infrastructure/Guides/meta.json | 2 +- .../Guides/protect-an-app-with-sso.mdx | 168 ++++++++++++++++++ .../cloud-infrastructure/flux.mdx | 3 +- .../cloud-infrastructure/index.mdx | 9 +- .../cloud-infrastructure/meta.json | 2 +- .../cloud-infrastructure/sso.mdx | 129 ++++++++++++++ 7 files changed, 312 insertions(+), 4 deletions(-) create mode 100644 content/docs/infrastructure/Guides/protect-an-app-with-sso.mdx create mode 100644 content/docs/infrastructure/getting-started/cloud-infrastructure/sso.mdx diff --git a/content/docs/infrastructure/Guides/deploy-a-new-app.mdx b/content/docs/infrastructure/Guides/deploy-a-new-app.mdx index 4cd318f..bafa89a 100644 --- a/content/docs/infrastructure/Guides/deploy-a-new-app.mdx +++ b/content/docs/infrastructure/Guides/deploy-a-new-app.mdx @@ -232,6 +232,9 @@ spec: Cloudflare terminates HTTPS, so the `Ingress` only needs the plain `web` entrypoint: no certificates in the cluster. +If only some people should open the app and it has no login of its own, put it behind the +SSO gateway: see [Protect an App with SSO](/docs/infrastructure/Guides/protect-an-app-with-sso). + ## Automatic image updates Flux can redeploy the app every time a new `latest` image is published, pinning the image digest (see [how it works](/docs/infrastructure/getting-started/cloud-infrastructure/flux#automatic-image-updates)). diff --git a/content/docs/infrastructure/Guides/meta.json b/content/docs/infrastructure/Guides/meta.json index 3704cc0..7844a2c 100644 --- a/content/docs/infrastructure/Guides/meta.json +++ b/content/docs/infrastructure/Guides/meta.json @@ -1 +1 @@ -{ "title": "Guides", "pages": ["add-a-new-secret", "add-storage", "backups", "deploy-a-new-app"] } +{ "title": "Guides", "pages": ["add-a-new-secret", "add-storage", "backups", "deploy-a-new-app", "protect-an-app-with-sso"] } diff --git a/content/docs/infrastructure/Guides/protect-an-app-with-sso.mdx b/content/docs/infrastructure/Guides/protect-an-app-with-sso.mdx new file mode 100644 index 0000000..eee7f03 --- /dev/null +++ b/content/docs/infrastructure/Guides/protect-an-app-with-sso.mdx @@ -0,0 +1,168 @@ +--- +title: "Protect an App with SSO" +--- + +## Introduction + +This guide puts an app behind the [SSO gateway](/docs/infrastructure/getting-started/cloud-infrastructure/sso): +visitors sign in with PoliNetwork Auth, and only people with a given permission get +through. Use it for apps without an OIDC login of their own. If the app supports OIDC, +give it its own client on Auth instead. + +It assumes the app is already deployed and published on a domain +(see [Deploy a New App](/docs/infrastructure/Guides/deploy-a-new-app)). In the examples, +the namespace is `` and the permission is `:access`. + +## Procedure + +1. [Create the permission](#create-the-permission) on Auth +2. [Add the Middleware](#add-the-middleware) +3. [Use it from the Ingress](#use-it-from-the-ingress) +4. [Admit only Traefik](#admit-only-traefik) +5. Open a PR into polinetwork-cd, wait for CI, merge +6. [Check it](#check-it), and only then disable the app's own login if you want to + +### Create the permission + +On [auth.polinetwork.org](https://auth.polinetwork.org), under *Access → Permissions*, +create a permission named `:access` (e.g. `uptime:access`). Then grant it to the roles +that should use the app, under *Access → Roles*. + +You need `idp:permissions:write` and `idp:roles:write` for this; ask an IdP Admin +otherwise. No new application is needed: every app behind the gateway shares the +*SSO Gateway* client. + +### Add the Middleware + +```yaml title="apps//middleware.yaml" +apiVersion: traefik.io/v1alpha1 +kind: Middleware +metadata: + name: sso +spec: + forwardAuth: + address: http://oauth2-proxy.sso.svc.cluster.local:4180/?allowed_groups=:access + trustForwardHeader: true + authResponseHeaders: + - X-Auth-Request-User + - X-Auth-Request-Email + - X-Auth-Request-Preferred-Username + addAuthCookiesToResponse: + - __Secure-pn_sso + - __Secure-pn_sso_0 + - __Secure-pn_sso_1 + - __Secure-pn_sso_2 + maxResponseBodySize: 1048576 +``` + +Change only the permission in `address`. The rest is the same for every app: + +- `trustForwardHeader` passes the `https` scheme and the host to oauth2-proxy, so visitors + come back to the right page after signing in. +- `authResponseHeaders` tells the app who is signed in: `X-Auth-Request-User` and + `X-Auth-Request-Email` both carry the person's email address. Traefik also deletes + these headers from incoming requests, so visitors can't forge them. Only trust them in + an app that the `NetworkPolicy` below protects. +- `addAuthCookiesToResponse` sends the refreshed session cookie back to the browser. + **Don't remove it**: without it, sessions break after about 4 minutes. + + + +`allowed_groups` takes a comma-separated list, and **any one** permission is enough +(`?allowed_groups=a:access,b:access`). For different rules on different paths, add one +Middleware per rule, with one `Ingress` each. + + + +Add the file to `apps//kustomization.yaml`, and make the app's Flux +Kustomization in `clusters/k3s/apps/.yaml` wait for the gateway: + +```yaml title="clusters/k3s/apps/.yaml" + dependsOn: + - name: infrastructure-secret-stores + - name: infrastructure-storage + - name: infrastructure-traefik + # [!code ++] + - name: infrastructure-sso +``` + +### Use it from the Ingress + +Reference the Middleware as `-@kubernetescrd`: + +```yaml title="apps//ingress.yaml" +metadata: + name: + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web + # [!code ++] + traefik.ingress.kubernetes.io/router.middlewares: -sso@kubernetescrd +``` + +The annotation applies to every host and path of that `Ingress`. Anything that must stay +public (a status page, a webhook, a health endpoint) goes in a **separate** `Ingress` +without the annotation, and that one should list exact paths, not `/`. See +`apps/uptime-kuma/ingress.yaml`: `uptime.polinetwork.org` needs `uptime:access`, while +`status.polinetwork.org` lists only the status page paths. + +### Admit only Traefik + +The gateway protects the route, not the pod. Add a `NetworkPolicy` so nothing else in the +cluster can call the app's Service directly: + +```yaml title="apps//network-policy.yaml" +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: allow-traefik +spec: + podSelector: + matchLabels: + app: + policyTypes: [Ingress] + ingress: + - from: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: kube-system + podSelector: + matchLabels: + app.kubernetes.io/name: traefik + ports: + - port: + protocol: TCP +``` + +If other pods legitimately call the app (e.g. Prometheus scraping it), add them as further +`from` entries. + + + +Turn off the app's built-in login only once all of the following hold. Otherwise anyone +who reaches the app is an admin. + +- The `NetworkPolicy` is applied. +- Every public `Ingress` lists only public paths. +- You have checked the gateway in a private window. + + + +### Check it + +In a private window: + +1. Open the app. You should land on `auth.polinetwork.org`, and after signing in, back on + the page you opened. +2. Sign in with an account **without** the permission: you should get `403 Forbidden`. +3. Stay on the app for more than 5 minutes and keep using it: you shouldn't be sent back + to Auth. + +## Troubleshooting + +| Symptom | Likely cause | +|---------|--------------| +| `403` for everyone | The permission isn't granted to your role, or its name differs from `allowed_groups`. Check *Access → Roles* on Auth. | +| Back to the app on `http://` after sign-in | `trustForwardHeader: true` is missing from the Middleware. | +| Sent back to Auth every few minutes | `addAuthCookiesToResponse` is missing or doesn't list `__Secure-pn_sso`. | +| `404` on the app | The Middleware name or namespace in the annotation is wrong: Traefik drops the route. Check Traefik's logs in `kube-system`. | +| `500` on every protected app | oauth2-proxy is down: check `kubectl get pods -n sso` and the `infrastructure-sso` Kustomization in the Flux Web UI. | diff --git a/content/docs/infrastructure/getting-started/cloud-infrastructure/flux.mdx b/content/docs/infrastructure/getting-started/cloud-infrastructure/flux.mdx index 230928e..d4aa23c 100644 --- a/content/docs/infrastructure/getting-started/cloud-infrastructure/flux.mdx +++ b/content/docs/infrastructure/getting-started/cloud-infrastructure/flux.mdx @@ -65,7 +65,8 @@ that is not listed there does nothing. |-----------|--------------| | `security` | Admission policies (no host namespaces, no pods as `keyvault-reader`) | | `storage` | [local-path provisioner](https://github.com/rancher/local-path-provisioner) and the `fast` / `standard` storage classes | -| `traefik` | Makes the bundled Traefik a `ClusterIP` service | +| `traefik` | Makes the bundled Traefik a `ClusterIP` service and trusts `X-Forwarded-*` from `cloudflared` | +| `sso` | The [SSO gateway](/docs/infrastructure/getting-started/cloud-infrastructure/sso) (oauth2-proxy) for apps without their own login | | `external-secrets` | [External Secrets Operator](https://external-secrets.io/), installed with Helm | | `secret-stores` | The `kv-pn-apps` and `kv-pn-infra` `ClusterSecretStore`s | | `cloudflared` | The Cloudflare Tunnel connector (2 replicas) | diff --git a/content/docs/infrastructure/getting-started/cloud-infrastructure/index.mdx b/content/docs/infrastructure/getting-started/cloud-infrastructure/index.mdx index 46e9965..13d8924 100644 --- a/content/docs/infrastructure/getting-started/cloud-infrastructure/index.mdx +++ b/content/docs/infrastructure/getting-started/cloud-infrastructure/index.mdx @@ -96,7 +96,9 @@ In summary, **Terraform builds the foundation on which applications run**, while `cloudflared` pods in the cluster. 3. `cloudflared` forwards it to **Traefik**, the K3s ingress controller. 4. Traefik picks the app from the `Ingress` whose `host` matches, and forwards the - request to that app's `Service`. + request to that app's `Service`. If the `Ingress` uses the SSO Middleware, Traefik + first asks the [SSO gateway](/docs/infrastructure/getting-started/cloud-infrastructure/sso) + whether the visitor is signed in and holds the app's permission. The VM never accepts inbound connections from the Internet: the tunnel is an outbound connection, and the VM's public IP is used only for outbound traffic. @@ -117,6 +119,11 @@ outbound connection, and the VM's public IP is used only for outbound traffic. href="/docs/infrastructure/getting-started/cloud-infrastructure/flux" description="How Flux deploys the polinetwork-cd repository to the K3s cluster." /> + ## References diff --git a/content/docs/infrastructure/getting-started/cloud-infrastructure/meta.json b/content/docs/infrastructure/getting-started/cloud-infrastructure/meta.json index 0bfe795..52c96bd 100644 --- a/content/docs/infrastructure/getting-started/cloud-infrastructure/meta.json +++ b/content/docs/infrastructure/getting-started/cloud-infrastructure/meta.json @@ -1 +1 @@ -{ "title": "Cloud Infrastructure", "pages": ["terraform", "k3s-node", "flux"] } +{ "title": "Cloud Infrastructure", "pages": ["terraform", "k3s-node", "flux", "sso"] } diff --git a/content/docs/infrastructure/getting-started/cloud-infrastructure/sso.mdx b/content/docs/infrastructure/getting-started/cloud-infrastructure/sso.mdx new file mode 100644 index 0000000..8df804f --- /dev/null +++ b/content/docs/infrastructure/getting-started/cloud-infrastructure/sso.mdx @@ -0,0 +1,129 @@ +--- +title: "SSO Gateway" +--- + +## Overview + +Some apps have no login of their own, or one we'd rather not maintain (e.g. Uptime Kuma). +For those, Traefik asks an **SSO gateway** whether each request may go through. The +gateway signs people in with [PoliNetwork Auth](https://auth.polinetwork.org) and lets +through only people who hold the permission that app requires. + +The gateway is [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/), deployed from +`infrastructure/sso` in [polinetwork-cd](https://github.com/PoliNetworkOrg/polinetwork-cd). +It replaces Cloudflare Access for HTTP apps. SSH to the node still goes through +Cloudflare Access for Infrastructure (see [Setup](/docs/infrastructure/getting-started/setup)). + + + +Apps that can sign people in with OIDC themselves (Grafana, the Flux Web UI, Vaultwarden, +our own apps) keep doing so with their own client on PoliNetwork Auth. The gateway is for +everything else. + + + +## How a request goes through + +```mermaid +sequenceDiagram + participant B as Browser + participant T as Traefik + participant P as oauth2-proxy (sso) + participant A as auth.polinetwork.org + participant K as App + B->>T: GET https://uptime.polinetwork.org/ + T->>P: forwardAuth: /?allowed_groups=uptime:access + alt no session + P-->>T: 302 to auth.polinetwork.org + T-->>B: 302 + B->>A: sign in + A-->>B: 302 to sso.polinetwork.org/oauth2/callback + B->>P: callback (code) + P-->>B: session cookie on .polinetwork.org, 302 back to the app + else session without the permission + P-->>T: 403 + T-->>B: 403 + else session with the permission + P-->>T: 202 (+ refreshed cookie) + T->>K: original request + K-->>B: response + end +``` + +1. The app's `Ingress` references a Traefik `forwardAuth` **Middleware**. Before routing + the request, Traefik sends its headers to oauth2-proxy at + `http://oauth2-proxy.sso.svc.cluster.local:4180/?allowed_groups=`. +2. Without a session, oauth2-proxy redirects to PoliNetwork Auth (authorization code + flow with PKCE). After sign-in, Auth sends the browser back to + `https://sso.polinetwork.org/oauth2/callback`, which sets the session cookie and + returns the visitor to the page they asked for. +3. With a session, oauth2-proxy checks the permission. It answers `202` and Traefik + forwards the request to the app, or `403` and the visitor sees an error page. + +All protected apps share **one** client on Auth (*SSO Gateway*) and **one** redirect URI. +Adding an app doesn't touch Auth's application list: it only needs a permission. + +## Permissions + +The Middleware names the permission in `allowed_groups`. oauth2-proxy reads it from the +`polinetwork_permission_list` claim, the array version of `polinetwork_permissions`, which +it requests with the `polinetwork:identity` scope. + +- It checks **permissions**, never roles. The `groups` claim holds role keys and must not + be used for access. +- By convention, each app has its own `:access` permission (e.g. `uptime:access`), + granted to the roles that should use it. Granting a role a permission is done on Auth, + not in Git. + +## Sessions + +- **The cookie.** The session lives in an encrypted, `HttpOnly`, `Secure` cookie named + `__Secure-pn_sso`, set on `.polinetwork.org`. One sign-in covers every protected host. + Large sessions are split into `__Secure-pn_sso_0`, `_1`, and so on. Nothing is stored + server-side, so the two oauth2-proxy replicas are interchangeable. +- **Refresh.** Auth's ID and access tokens last 5 minutes. After 4 minutes, the next + request makes oauth2-proxy use the refresh token (`offline_access`) to get new ones, and + re-read the permissions. Losing a permission or a role takes effect within about 5 + minutes, without signing out. +- **Rotation.** Auth rotates the refresh token on every use, so the refreshed cookie + **must** reach the browser. That is what `addAuthCookiesToResponse` in the Middleware + does. Without it, the browser keeps sending a refresh token that has already been used. +- **Lifetime.** The cookie lasts 24 hours. After that, the visitor goes through Auth + again, usually without typing anything if their Auth session is still active. +- **Sign-out.** `https://sso.polinetwork.org/oauth2/sign_out` deletes the cookie. It does + not end the session on Auth. + +## Security notes + +- **Only Traefik may reach a protected app.** The gateway protects the route, not the + pod. Every app behind it has a `NetworkPolicy` that admits only Traefik, or another pod + in the cluster could call its Service directly. This matters most for apps whose own + login is disabled. +- **Everything else on the app's host is public.** If a host serves both a private and a + public part, as Uptime Kuma does with its status page, the public part needs its own + `Ingress` that lists only the public paths. +- **It fails closed.** If oauth2-proxy is down, protected apps return errors; they don't + open up. +- **The cookie is a bearer credential.** Browsers send it to every `*.polinetwork.org` + host, including apps not behind the gateway. That is the price of one sign-in for all + apps; every host on the domain is ours. +- **Forwarded headers.** Traefik's `web` entrypoint trusts `X-Forwarded-*` from the pod + network, where `cloudflared` runs. oauth2-proxy uses `X-Forwarded-Proto` and + `X-Forwarded-Host` to send visitors back to `https:///...`, and only to + `.polinetwork.org` hosts (`--whitelist-domain`). +- **Rate limits.** Refreshes call Auth's token endpoint from inside the cluster, so they + count against Auth's shared budget of 60 cluster-internal token requests per minute. + That's plenty for staff dashboards; don't put a high-traffic public app behind the + gateway. + +## Where things live + +| What | Where | +|------|-------| +| oauth2-proxy Deployment, Service, Ingress, NetworkPolicy | `infrastructure/sso/` | +| Client secret and cookie secret | Key Vault `kv-pn-infra`: `sso-oidc-client-secret`, `sso-cookie-secret` | +| The *SSO Gateway* client | [auth.polinetwork.org/applications](https://auth.polinetwork.org/applications) | +| Trusted forwarded headers | `infrastructure/traefik/traefik-service.yaml` | +| Per-app Middleware (example) | `apps/uptime-kuma/middleware.yaml` | + +To put an app behind the gateway, follow [Protect an App with SSO](/docs/infrastructure/Guides/protect-an-app-with-sso).