Skip to content

[SDK Sentinel] Fix Nexus TRY_CANCEL cancellation test race - #1785

Open
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/python/32792182767-1-c5cfe6f1316a
Open

[SDK Sentinel] Fix Nexus TRY_CANCEL cancellation test race#1785
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/python/32792182767-1-c5cfe6f1316a

Conversation

@sdk-sentinel-bot

Copy link
Copy Markdown
Contributor

Caution

This PR contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.

Summary

Prevent the Nexus TRY_CANCEL cancellation test from hanging. Two Linux ARM main-branch jobs and a matching PR job timed out; the [earliest captured main failure](https://github.com/temporalio/sdk-python/actions/runs/32746977318) is the first bounded occurrence. No preceding equivalent success was captured.

Root cause

The cancel handler released its backing workflow and then returned an error. Workflow completion could therefore race cancel-failure recording. If completion won, the test waited indefinitely for the losing cancel-request-failed transition.

Fix

Keep the handler workflow running until cancel-request failure is visible, then release it. This makes the tested ordering explicit while preserving the result and history assertions.

Validation

The exact TRY_CANCEL case passed 20 stress iterations after the change, and all four cancellation modes passed once. The unchanged case also passed 20 times, so natural reproduction was not observed. The trusted baseline passed. Candidate Python lint passed, but bridge lint and the editable build require CI because this environment had no active Rust toolchain.

Validation status: validation-incomplete

  • Flake confidence: high
  • Fix confidence: medium
  • Value: high
  • Patch scope: standard — 1 changed file; 1.8 KiB
  • Local reproduction: not-reproduced — The unchanged TRY_CANCEL case passed 20 local stress iterations; the CI-only ordering remained rare.
  • Regression coverage: pass-after-only — The existing exact test passed 20 stress iterations after the ordering fix, and all cancellation modes passed once; no new test was added.
  • Unchanged baseline (Poe lint, bridge lint, and editable build) — passed
  • Independent candidate (Poe lint, bridge lint, and editable build) — passed

Investigator-run checks

  • poe test -s -n 0 --flake-finder --flake-runs=20 -k 'test_cancellation_type and TRY_CANCEL' — passed; 20 attempt(s). Before the change, the target case passed 20 local iterations; natural reproduction was not observed.
  • poe test -s -n 0 tests/nexus/test_workflow_caller_cancellation_types_when_cancel_handler_fails.py -k TRY_CANCEL --flake-finder --flake-runs=20 — passed; 20 attempt(s). The patched TRY_CANCEL case passed all 20 iterations.
  • poe test -s -n 0 tests/nexus/test_workflow_caller_cancellation_types_when_cancel_handler_fails.py — passed; 1 attempt(s). All four cancellation modes passed.
  • poe test -s -k 'test_temporal_operation_update_workflow and not delayed' — passed; 1 attempt(s). The separate recurring Nexus update timeout did not reproduce.
  • trusted unchanged baseline validation — passed; 1 attempt(s). The supplied baseline reports successful lint, bridge lint, and editable build.
  • .ci-flake-runtime/input/validate.sh — failed; 2 attempt(s). Python lint and type checks passed, but cargo clippy stopped because rustup had no active toolchain; build-develop was not reached.

Required target CI

  • Continuous Integration / build-lint-test (3.14, ubuntu-arm)
  • Fixed repository validation with the configured Rust toolchain: poe lint, poe bridge-lint, and poe build-develop

Residual risks

  • The original ordering was not reproduced locally, so CI must confirm the candidate under Linux ARM load.
  • Bridge lint and editable build remain unverified in the candidate environment because no Rust toolchain was active.

Automation provenance

If this finding should not be fixed, apply the sdk-sentinel:false-positive label and close the PR. Sentinel will suppress the finding until its affected source changes.

SDK Sentinel recurrence history

  • Unique failed CI run attempts: 1
  • First occurrence: 2026-08-24 16:00 UTC
  • Latest occurrence: 2026-08-24 16:00 UTC
  • Recent occurrences:

@sdk-sentinel-bot
sdk-sentinel-bot requested a review from a team as a code owner August 25, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant