Skip to content

feat(pii): env-driven uvicorn worker count (PII_WORKERS) #5457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TheodoreSpeaks merged 2 commits into from
Jul 7, 2026
+11 −4
Merged

feat(pii): env-driven uvicorn worker count (PII_WORKERS) #5457

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
65878cf
feat(pii): env-driven uvicorn worker count (PII_WORKERS)
TheodoreSpeaks Jul 7, 2026
044eb3d
harden(pii): quote PII_WORKERS expansion + raise healthcheck start-pe…
TheodoreSpeaks Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions docker/pii.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,16 @@ USER pii
# 5001 avoids colliding with the app's 3000 in local/compose runs on one host.
EXPOSE 5001

# start-period is generous: five large spaCy models load at import before
# /health responds. Tune against measured cold-start once built.
HEALTHCHECK --interval=30s --timeout=5s --start-period=180s --retries=3 \
# start-period covers the model cold start. With PII_WORKERS>1 each worker loads
# the five spaCy models independently and in parallel, so allow generous headroom
# (memory-bandwidth contention stretches the wall-time beyond the single-worker case).
HEALTHCHECK --interval=30s --timeout=5s --start-period=300s --retries=3 \
CMD curl -fsS http://localhost:5001/health || exit 1

CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "5001"]
# Worker count is env-driven so ONE image scales per task size: set PII_WORKERS to
# the task's vCPU count (each worker loads the models independently, ~3 GB each, so
# size task memory ≈ PII_WORKERS × 3 GB + overhead). Defaults to 1 for local/small.
# `sh -c exec` expands the env var while keeping uvicorn as PID 1 for clean SIGTERM.
# Quote the expansion so a malformed PII_WORKERS fails uvicorn arg-parsing rather
# than being interpreted by the shell.
CMD ["sh", "-c", "exec uvicorn server:app --host 0.0.0.0 --port 5001 --workers \"${PII_WORKERS:-1}\""]

@cursor cursor Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supervisor duplicates spaCy model RAM

High Severity

With PII_WORKERS greater than one, the new --workers flag turns on uvicorn’s multiprocess supervisor. The image’s pinned uvicorn 0.49.0 eagerly imports server:app in that supervisor before workers start, so the five spaCy models load once in the parent and again in each worker. Task memory sized as PII_WORKERS × ~3 GB can OOM or thrash during rollout.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 044eb3d. Configure here.

@TheodoreSpeaks TheodoreSpeaks Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is fine. I can tune this manually.

Loading