fix(swr): release refresh slots stranded on stopped event loops (LAB-7295)

docs/configuration.md

@@ -195,6 +195,7 @@ Rules and behavior:
 - Requires a positive `ttl`; `ttl + stale_ttl` is capped at 2,592,000 s (30 days). Violations raise `ConfigurationError` at decoration time.
 - **CachekitIO only, known at decoration time** — `@cache.io` or an explicit `backend=CachekitIOBackend()`. Other backends have no read-side freshness signal and raise `ConfigurationError` if `stale_ttl` is set; so does a CachekitIO backend resolved lazily from `CACHEKIT_API_KEY` under another preset (the remaining-freshness bound below still applies to its reads).
 - Concurrent stale hits trigger at most one revalidation: per-process dedup plus (async functions) a non-blocking distributed lease on the backend's lock. Contested = serve stale, don't wait.
+- At most 32 revalidations per decorated function run at once. A stale hit that finds the limit reached serves stale without revalidating, and a later hit retries. An async revalidation stranded on a stopped event loop (sync code that calls `loop.run_until_complete` and leaves the loop stopped, or closes it without cancelling its tasks) gives up its turn 30 seconds after it was scheduled and is cancelled, so a stale hit on another loop can revalidate that key again. If its loop resumes, it stores nothing, even when your function swallows the cancellation and returns. A write it had already sent before its loop stopped is not recalled, and can land after a newer revalidation's.
 - A failed background recompute never reaches the caller: the entry keeps serving stale until its hard eviction bound, after which the next call takes the ordinary synchronous miss path. It logs a WARNING `SWR revalidation failed`, as does a revalidation skipped because the call's arguments cannot be deep-copied (`SWR revalidation skipped`) or one that could not be scheduled. It names the function by the same digest as the L1-only refresh WARNING. Each fires at most once a minute per function and carries the count since the last one; the occurrences in between log at DEBUG.
 - The background recompute runs with a **snapshot of the caller's `contextvars`** (contextvar-based tenant extraction works), but outside the request otherwise — don't rely on other request-scoped resources (open sessions, connections) inside functions that enable SWR.
 - Stale values are never written to the L1 in-memory cache, and stale reads still count as cache **hits** for metered-misses billing.
@@ -304,6 +305,14 @@ honored as follows:
   exception. Set `swr_retry_interval=0` to retry on every stale hit. The refresh runs on a deep copy
   of the call's arguments; when they cannot be copied (a lock, an open connection), it is
   skipped, so that call is only ever recomputed in the foreground after expiry.
+- **At most 32 background refreshes run at once per function.** A stale hit that finds the
+  limit reached serves the cached value without refreshing, and a later stale hit retries.
+  An `async def` refresh stranded on a stopped event loop (sync code that calls
+  `loop.run_until_complete` and leaves the loop stopped, or closes it without cancelling its
+  tasks) gives up its turn 30 seconds after it was scheduled and is cancelled. The next
+  refresh attempt on another key of that function releases it; from then on stale hits on
+  other loops refresh that key again. Until then the key keeps serving its cached value,
+  and past its `ttl` the next call recomputes it in the foreground.
 - **Failed and skipped refreshes log a WARNING** (`L1-only SWR refresh failed`, `… skipped`,
   `… could not be started`) with the redacted function, the redacted key and the exception
   type. The function appears as the `<redacted:…>` digest of its `module.qualname`;

docs/free-threading.md

@@ -55,7 +55,7 @@ the ticket, plus what the free-threaded CI lane surfaced and sites added since:
 | `os.register_at_fork` handlers (session + stats registry) | Run in the child while single-threaded; replace locks wholesale | Safe — single-threaded by construction at execution time |
 | `backends/cachekitio/session.py` header cache | `threading.local` | Safe — per-thread state; its only cross-thread hazard was the session-identity publication above |
 | `decorators/stats_context.py` | `contextvars.ContextVar` | Safe by construction |
-| `decorators/wrapper.py` L1/SWR + L2/SWR single-flight (`_l1_swr_*`, `_l2_swr_*`) | `BoundedSemaphore` slots + in-flight `set` + PID-owner swap | Safe. The check-then-add on the in-flight set was already documented as benign (worst case one duplicate refresh, absorbed by last-write-wins / the backend lease); builtin `set`/`dict` single ops are atomic under free-threading's per-object locking. The fork-detection wholesale swap races are the same documented-benign shape |
+| `decorators/wrapper.py` L1/SWR + L2/SWR single-flight (`_RefreshPool`: `_l1_swr_pool`, `_l2_swr_pool`) | One `Lock` per pool over its holder map (an L2 holder carries its in-flight key) + PID-owner swap | Safe. The prune, the cap check and L2's per-key check run under the pool lock, so L2 has no check-then-add race; L1's per-key check is `ObjectCache`'s, under its own lock. A pruned refresh is cancelled with `call_soon_threadsafe` on its own loop. The fork-detection swap race is the documented-benign shape: a thread racing it can lose its new holder, whose key then waits for its hard TTL (LAB-7295) |
 | `l1_cache.py` `L1CacheManager._take_over_if_forked` + `_empty_caches_after_fork` (LAB-4772) | Owner-PID fast path; on a PID change one thread takes over under a lock fetched by `dict.setdefault(pid, Lock())`. An `os.register_at_fork` hook gives every cache a fresh, empty state before the child's first `get()`, so a forked child starts with an empty L1; fork mechanics and the uWSGI limit are in the `_take_over_if_forked` docstring | Safe. `setdefault` is atomic under per-object locking, so every thread in a new child gets the same fresh lock, which no parent thread can have held at fork; the rest block on it and re-check. The fast path gates on `_owner_pid` alone, so a weak-memory reader may still see the inherited manager lock once: the same benign fork-swap race as the SWR row, since that lock is unheld unless a parent thread held it at fork. The hook runs while the child is single-threaded and replaces every cache's state, held lock or not, so it probes no lock, takes none and starts no thread. It does not free the inherited states, which would copy the parent's L1 pages into every child inside `fork()`: they wait in `_inherited_states` until the child's first put, whose take-over frees them in batches, giving up the GIL between batches, on the cleanup thread it restarts or, when the parent ran none, on a one-shot thread. Each state is taken off that list with one `list.pop()`, atomic under per-object locking, before it is freed, so two releasers running at once (the threads of two managers' take-overs) never take the same state and neither finds the list emptied under it. So a child bound for exec never pays. A child that uses its L1 pays once, up front: each free also writes the allocator's bookkeeping beside the entry, so its first put copies about 2.3 times the inherited L1 (36.5 MiB for a 16 MiB L1 in `tests/performance/test_l1_fork_memory.py`), and its own L1 then reuses that memory. Kept, those pages would cost nothing while the parent left them alone, as an idle preloaded master whose entries outlive its workers does. Freeing is the default anyway because a parent with a cleanup thread rewrites them within one L1 TTL, and from then on a child that kept them would hold a private copy nothing can reach, beside its own L1. A state whose lock is taken is only dropped: one a thread forked inside its critical section holds is freed with that thread's last reference, and one a parent thread held at fork can stay for the child's life, referenced by that thread's frame. Neither the hook nor the take-over clears shared state: each publishes a fresh `_L1State` (entries, byte count and lock) in one store, and a holder that is still running finishes on the state it bound (the forking thread, if it forked inside a critical section; without hooks, a live child thread holding a lock past the take-over's 1 s probe). Invalidations re-apply if a reset replaced the state meanwhile. Without hooks, the cost of a wrong guess is that namespace's L1 entries, once per child; `--py-call-osafterfork` avoids it. Once it has run, the take-over skips its own timed cache-lock probe, which could otherwise drop a cache's entries under a live child thread. In a child the hook never reached (a fork made from C, as uWSGI does without `--py-call-osafterfork`; any PID other than the importing process's and the one the hook last ran in), the take-over starts no thread and `start_background_cleanup` refuses to start one, including for a manager first built in that child: such a fork also skips CPython's own after-fork repair, so a thread started there can hang in `Thread.start()` or crash the interpreter. None of that logs, the take-over's lock-reset drop included, because the same fork skips `logging`'s at-fork lock reset, and a handler lock a parent thread held would hang the child. Expired entries in that child are evicted on read or under the memory bound instead. The detection needs cachekit imported before the fork: a process that first imports it after a fork made from C (uWSGI `--lazy-apps` without `--py-call-osafterfork`) looks like a fresh process, and starts the cleanup thread. No check made at import tells the two apart |
 | `backends/cachekitio/client.py` client caches + `CachekitIOBackend._own_sync_lease` / `_own_async_lease` | Per-thread `threading.local` caches of httpx clients, each thread's carrying the PID that filled them; each lease records the PID that built it, and the backend re-leases on a mismatch, checked per request | Safe across fork. A forked child never sends on or reads from its parent's pooled connections, which share the parent's TLS sessions: its first request re-leases and builds new clients, whether it was forked by `os.fork()` or from C with no at-fork hooks (uWSGI). An owner-PID check rather than an at-fork hook for that reason. The backend's check reads one reference, the lease, that carries its own PID (single-assignment publication), so no thread pairs a new PID with an inherited client. A thread's caches and their PID are read and reset by that thread alone, so no thread can discard another's. Inherited clients are dropped unclosed: `close()` takes the pool lock, which a parent thread may have held at fork, and garbage collection closes the child's copies of the sockets without sending anything. Two child threads racing the first request may each build a client; the one replaced is simply not shared, and each request holds its lease until it returns, so that replacement never closes a client under an in-flight request. A new async lease also drops the per-loop slot, so a child still running its parent's event loop object gets a new client. A re-lease takes no `logging` lock while the `hpack` logger keeps the level the parent pinned when it built a client (the logger is looked up at import), because a fork from C also skips `logging`'s at-fork lock reset. That covers cachekit's own path only: in such a child, any logging call whose level check is not already cached, httpx's per-request log included, waits on `logging`'s module lock, and hangs if a parent thread held it at fork. A logging reconfiguration clears those caches, so a fork from C made after one, while a thread is inside `logging`, can hang the child's first request, with or without this re-lease. If that reconfiguration unset `hpack`'s level and the parent has opened a connection since, which caches those checks again, the re-lease itself hangs: it pins the level again with `setLevel`, which takes the lock. The pin stays, because both lock-free alternatives log the bearer key: skipping it leaves the child unpinned, and writing the level without `setLevel` leaves a cached DEBUG check enabled. `--py-call-osafterfork` avoids all of these. Tests in `tests/unit/backends/test_cachekitio_fork.py` |
 | `invalidation.py` dispatch map, start lock and listener thread | `_evictors` (registry id → `WeakSet` of evictors) under a per-PID `Lock` from `dict.setdefault` (`_dispatch_locks`): decoration registers under it, and the listener thread takes a snapshot under it and runs the evictors outside it. The listener is redis-py's own `PubSubWorkerThread`, started behind an owner-PID check under a second per-PID lock (`_start_locks`) taken non-blocking, so a cache operation never waits on another thread's start | Safe. `setdefault` is atomic under per-object locking, so threads racing to create a new process's lock share one, and a child forked while a parent thread holds either lock gets its own instead of waiting on a lock nobody will release. Walking the live `WeakSet` while a decoration adds to it would raise; the snapshot under the lock rules that out. The owner-PID check reads `_listener_pid`, which the starting thread publishes last, so a stale read only skips or retries a start. Evictors call `L1Cache.invalidate` and `invalidate_many`, which lock per state as in the row above. A child forked without at-fork hooks starts no listener and logs nothing (the same detection as the row above). Tests in `tests/unit/test_invalidation_listener.py` |