bug: PKI certgen hardcodes namespace "openshell" in server certificate SANs — breaks deployments in other namespaces

[akram]

Agent Diagnostic

• Explored crates/openshell-bootstrap/src/pki.rs:32-42 — DEFAULT_SERVER_SANS hardcodes openshell.openshell.svc and openshell.openshell.svc.cluster.local regardless of the deployment namespace.
• Explored deploy/helm/openshell/templates/certgen.yaml:90-112 — the certgen job has POD_NAMESPACE from the downward API but does not use it for SAN generation. It passes pkiInitJob.serverDnsNames (empty by default) as --server-san args, so only the Rust defaults apply.
• Explored deploy/helm/openshell/values.yaml:340-347 — certManager.serverDnsNames also hardcodes the openshell namespace.
• Verified by deploying in namespace openshell-system: the cert SANs contain openshell.openshell.svc.cluster.local but the service FQDN is openshell.openshell-system.svc.cluster.local. Sandbox supervisors fail with BadCertificate.
• Confirmed the CA and client certs are correctly signed — the issue is purely SAN mismatch.

Description

The built-in PKI init job generates server TLS certificates with SANs that hardcode the namespace openshell. When the chart is installed in any other namespace (e.g., openshell-system, sandbox-prod), the server certificate SANs don't match the actual Kubernetes service FQDN. Sandbox supervisors connect to openshell.<actual-namespace>.svc.cluster.local but the cert only covers openshell.openshell.svc.cluster.local, causing a TLS handshake failure.

The same hardcoded SANs appear in certManager.serverDnsNames in values.yaml, so the cert-manager path has the same problem.

Reproduction Steps

1. Install the chart in a namespace other than openshell:

   helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
     --version 0.0.70 -n openshell-system

2. Create a sandbox: openshell sandbox create
3. The sandbox pod starts but the supervisor fails to connect to the gateway
4. Check gateway logs for BadCertificate
5. Inspect the cert SANs:

   oc get secret openshell-server-tls -n openshell-system \
     -o jsonpath='{.data.tls\.crt}' | base64 -d | \
     openssl x509 -text -noout | grep -A2 "Subject Alternative"

Expected: SANs include openshell.openshell-system.svc.cluster.local.

Actual: SANs are hardcoded to openshell.openshell.svc.cluster.local.

Environment

• OS: macOS 15.x (Apple Silicon)
• OpenShift: ROSA HCP 4.21 (Kubernetes 1.34.2)
• OpenShell: v0.0.70 chart
• Deployed in namespace openshell-system

Logs

ERROR openshell_server: TLS handshake failed error=received fatal alert: BadCertificate client=10.130.0.224:57486
ERROR openshell_server: TLS handshake failed error=received fatal alert: BadCertificate client=10.130.0.224:57488

Sandbox supervisor logs:

WARN openshell_sandbox: Policy fetch failed, retrying
Error: Policy fetch failed after 5 attempts: failed to connect to OpenShell server

Affected Code

Location	Issue
crates/openshell-bootstrap/src/pki.rs:32-42	DEFAULT_SERVER_SANS hardcodes openshell.openshell.svc
deploy/helm/openshell/templates/certgen.yaml:106-111	Passes pkiInitJob.serverDnsNames (empty by default) without namespace-aware defaults
deploy/helm/openshell/values.yaml:340-347	certManager.serverDnsNames hardcodes openshell.openshell.svc

Proposed Fix

Option A — Rust certgen (recommended): Read POD_NAMESPACE at runtime and generate namespace-aware SANs instead of the hardcoded constant. Skip DEFAULT_SERVER_SANS when --server-san args are provided.

Option B — Helm templates: Generate namespace-aware default SANs in the Helm template using .Release.Namespace and pass them as --server-san args. Implemented in PR #2062.

Workaround

Deploy in the openshell namespace, or use PR #2062 which fixes the Helm side.

---

• I pointed my agent at the repo and had it investigate this issue
• I loaded relevant skills (e.g., debug-openshell-cluster, debug-inference, openshell-cli)
• My agent could not resolve this — the diagnostic above explains why