Skip to content

WIP: Reduce shutdown delay duration in apiserver - #7514

Draft
pacevedom wants to merge 2 commits into
openshift:mainfrom
pacevedom:shutdown-delay-duration
Draft

pacevedom wants to merge 2 commits into
openshift:mainfrom
pacevedom:shutdown-delay-duration

Conversation

@pacevedom

@pacevedom pacevedom commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • Improvements
    • The kube-apiserver shutdown delay is now one second instead of five, shortening the wait during shutdown. This changes the timing of the shutdown process without altering the available shutdown controls.

@pacevedom

Copy link
Copy Markdown
Contributor Author

/test ?

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 8, 2026
@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

We couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting @coderabbitai full review.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Central YAML (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cdd4531f-7425-4cce-ae75-c9e3049197e8
📥 Commits

Reviewing files that changed from the base of the PR and between 2eccfe1 and b100bd7.

📒 Files selected for processing (1)
  • pkg/controllers/kube-apiserver.go

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.


Walkthrough

The kube-apiserver shutdown-delay-duration argument changes from 5s to 1s.

Changes

Shutdown delay

Layer / File(s) Summary
Update shutdown delay
pkg/controllers/kube-apiserver.go
The configured kube-apiserver shutdown delay changes from 5s to 1s.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~2 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to b100b

The one-second shutdown delay is not shown to violate a repository-defined traffic-drain requirement, so no concrete merge-blocking risk is established.

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: reducing the kube-apiserver shutdown delay duration. The WIP: prefix does not obscure the change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The pull request changes only the kube-apiserver shutdown-delay-duration value from 5s to 1s. It does not add or modify any Ginkgo test title, so it introduces no unstable or overly specific tes…
Test Structure And Quality ✅ Passed PASS: The pull request changes only the kube-apiserver configuration value from 5s to 1s. It adds or modifies no Ginkgo tests, so the stated test-structure requirements do not apply.
Microshift Test Compatibility ✅ Passed PASS: The pull request changes only pkg/controllers/kube-apiserver.go, reducing shutdown-delay-duration from 5s to 1s. It adds no Ginkgo e2e tests, so the MicroShift test compatibility check d…
Single Node Openshift (Sno) Test Compatibility ✅ Passed The pull request changes only the kube-apiserver shutdown delay from 5s to 1s. It adds no Ginkgo e2e tests, so the SNO compatibility check does not apply.
Topology-Aware Scheduling Compatibility ✅ Passed The pull request changes only the kube-apiserver argument shutdown-delay-duration from 5s to 1s in pkg/controllers/kube-apiserver.go. It does not add or modify deployment scheduling constraint…
Ote Binary Stdout Contract ✅ Passed The pull request changes only the kube-apiserver shutdown-delay-duration value from 5s to 1s. It adds no stdout write and does not modify main, init, TestMain, suite setup, or RunSpecs s…
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The pull request changes only pkg/controllers/kube-apiserver.go, reducing shutdown-delay-duration from 5s to 1s. It adds no Ginkgo e2e tests and introduces no IPv4 assumptions or external conn…
No-Weak-Crypto ✅ Passed The pull request changes only shutdown-delay-duration from 5s to 1s in pkg/controllers/kube-apiserver.go. The diff introduces no MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB mode, custom cryptogra…
Container-Privileges ✅ Passed The pull request changes only shutdown-delay-duration from 5s to 1s in pkg/controllers/kube-apiserver.go. It does not add or modify privileged, host namespace settings, SYS_ADMIN, or `allo…
No-Sensitive-Data-In-Logs ✅ Passed The pull request changes only the kube-apiserver shutdown-delay-duration value from 5s to 1s. It adds no logging and does not expose passwords, tokens, API keys, PII, session IDs, hostnames, or …
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@openshift-ci

openshift-ci Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@pacevedom

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-tests-bootc-el9

@openshift-ci

openshift-ci Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pacevedom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 8, 2026
pacevedom and others added 2 commits October 8, 2026 16:23
shutdown-delay-duration tells the apiserver how long to keep serving
after receiving SIGTERM before it stops accepting new requests. Its value
is a plain time.Sleep with no short-circuit, so it is spent in full on
every shutdown. MicroShift's graceful shutdown budget is 15s, so a 5s
delay consumed a third of it before any teardown work began.

That delay exists in multi-node OpenShift to let load balancers and
endpoint controllers route clients to another apiserver (the upstream
default is 70s). Single-node MicroShift has no second apiserver and no HA
load balancer to drain to, so the delay provides little benefit while
eating budget that the apiserver's own teardown needs. Reduce it to 1s,
which keeps a small graceful window (clients still get a Retry-After)
while reclaiming ~4s per shutdown. Typical stops drop from ~8s to ~3s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The kube-apiserver's post-delay teardown (in-flight drain, HTTP server
shutdown, post-shutdown hooks) can occasionally take ~16-18s under CPU
contention, exceeding the 15s graceful shutdown budget. When that happens
MicroShift force-exits mid-shutdown and logs "MICROSHIFT STOP TIMED OUT",
which trips the standard2 Log Scan test's "Services Should Not Timeout
When Stopping" check and causes intermittent CI failures.

The timeout is a ceiling, not a fixed wait: clean shutdowns still exit as
soon as all services stop (~3s with the 1s shutdown-delay-duration), so
this only gives the rare slow teardown room to finish gracefully instead
of being killed. 30s stays well under systemd's default TimeoutStopSec,
so there is no risk of a premature SIGKILL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@pacevedom
pacevedom force-pushed the shutdown-delay-duration branch from b100bd7 to 3a39dfd Compare October 8, 2026 14:24
@pacevedom

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-tests-bootc-el9

@openshift-ci

openshift-ci Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@pacevedom: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant