🐛 Fix intermittent deadlock in remeshing_im - #94
Draft
jdumas wants to merge 1 commit into
Draft
Conversation
jdumas
force-pushed
the
jdumas/repro-remeshing-im-deadlock
branch
from
September 8, 2026 14:36
7f2d797 to
76ca89b
Compare
Optimizer::optimizeOrientations() and optimizePositions() set the mOptimizeOrientations / mOptimizePositions predicate flags without holding mRes.mutex(), while the optimizer worker reads those same flags under that mutex in Optimizer::run(). This opens a lost-wakeup window. The worker evaluates its wait predicate while holding mRes.mutex(), then calls mCond.wait(), which only acquires the condition variable's internal mutex a few instructions later. If the main thread sets a flag and calls notify_all() inside that gap, the notification is delivered before the worker registers as a waiter and is lost. The worker then sleeps forever, and the main thread's Optimizer::wait() sleeps waiting for a flag the worker will never clear. Setting the flags while holding mRes.mutex() closes the window, since the worker holds that same mutex continuously from its predicate check until cv_any::wait() releases it. ThreadSanitizer reports the race between Optimizer::run() and Optimizer::optimizeOrientations() on an unfixed build and reports none after this change. ordered_lock is a non-recursive ticket lock and the flag setters take no lock themselves, so holding the mutex across these calls introduces no self-deadlock. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jdumas
force-pushed
the
jdumas/repro-remeshing-im-deadlock
branch
2 times, most recently
from
September 8, 2026 20:41
96a5f85 to
58e6aaf
Compare
Contributor
|
I have incorporated this fix upstream (+ a few other tsan/asan fixes): qnzhou/instant-meshes-core#5 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The
windows-2025 (Debug)job of #93 failed with:The ctest log shows this is a hang, not a slow test:
Section 1 finished in ~15s, so a proportionally-slow run would total ~50s. Instead there is zero
output for the full remaining 1500s. Note this job runs no sanitizer, so sanitizer overhead is
not involved.
Root cause
Optimizer::optimizeOrientations()/optimizePositions()set themOptimizeOrientations/mOptimizePositionspredicate flags without holdingmRes.mutex(), while the optimizer workerreads those same flags under that mutex in
Optimizer::run():This opens a lost-wakeup window:
mRes.mutex()and decides to sleep.mCond.wait(), which only acquires the condition variable's internal mutex a fewinstructions later.
notify_all()inside that gap, the notification isdelivered before the worker registers as a waiter, and is lost.
Optimizer::wait()sleeps waiting for aflag the worker will never clear. Deadlock.
The main thread can only slip into that gap because it writes the flag without the mutex. Note
that
condition_variable_anyis otherwise robust here: its internal mutex serialisesnotify_all()against waiter registration. The window is only a handful of instructions wide, which is why this
reproduces so rarely.
Fix
instant-meshes-coreis an external pinned dependency, so the fix is caller-side inremesh.cpp:set the optimizer flags while holding
mRes.mutex(). That closes the window, because the workerholds the same mutex continuously from its predicate check until
cv_any::wait()releases it.ordered_lockis a non-recursive ticket lock and the flag setters take no lock themselves, soholding the mutex across these calls introduces no self-deadlock.
Validation
The full 20-job matrix passed on this branch, including the
ubuntu-24.04 (gcc, Debug, ThreadSanitizer)job that previously timed out inxatlas(that one wasan unrelated lost-wakeup hang in the xatlas task scheduler, fixed by the dependency bump in #93):
Brute-force repetition did not reproduce the hang, as expected given how narrow the window is:
windows-2025 (Debug)in CI: all passed, very consistently at~15.3s each.
ThreadSanitizer does pin it down deterministically, on the exact pair of accesses in the analysis
above. Against an unfixed build:
4 such warnings before the change, 0 after it.
Note that
.github/tsan.suppressions.inicurrently hides this behind a broadrace:instant_meshes::entry, which already notes that Instant Meshes "has other data races in its
Optimizerclass" — sothe TSan CI job would not have caught this. Narrowing that suppression would be a reasonable
follow-up, but it likely surfaces other pre-existing races in the same library.
Known remaining issue
Optimizer::shutdown()has the same unsynchronised pattern:That one cannot be fixed from the call site, since the
join()is internal to the optimizer. Itneeds an upstream patch to
instant-meshes-core.