fix: a baseline that disappears mid-test raises instead of passing green (#217)

[pftg]

Closes the 2.0 subset of #217: the one silently wrong finding from the parallel-execution audit.

The bug

Every artifact path derives from the screenshot name alone. Two concurrently-running tests asserting the same name therefore race on one set of files: the winner's archive_baseline! moves the baseline the loser just checked out, and the loser's need_to_compare? then sees no baseline, records the screenshot as new, and returns without comparing. The test passes green having verified nothing. The audit measured 18–34 skipped comparisons out of 64.

The fix

"No baseline was ever committed" is a legitimate state, already warned about (#255). "The baseline I just checked out has disappeared" is impossible in a correct run, and was indistinguishable from the first at the point where it matters.

• Snap#checkout_base_screenshot records on the session that git really handed it a baseline.
• AssertionRegistry#record_new_screenshot raises for a name it knows had one, instead of recording it as never-committed.

The record is per-session (fiber-local — checkout and comparison happen in one call stack) and cleared by reset, since the session outlives the test.

The baseline for 'dashboard' was checked out and then disappeared before it could be compared -- nothing was verified.
Every artifact path derives from the screenshot name alone, so two tests asserting 'dashboard' at the same time race on one set of files: the one that finishes first archives the baseline the other is still using.
Give those screenshots distinct names, or do not run them concurrently.

Additive only — no behaviour change for any correct run (ADR-010: 2.0 breaks nothing).

Evidence

test/unit/baseline_disappeared_test.rb — two free-running threads, one name, no injected deletion:

	silent skips
before the fix (5 runs of 40 assertions)	0, 1, 1, 1, 1 — reproduced in 4 of 5
after the fix (10 runs)	0 of 10

The audit's other, already-loud failure mode (the baseline vanishing after need_to_compare?, giving a Vips::Error on a missing file) still appears in the tally and is untouched — it was never silent.

Mutation transcript (each restored with a targeted edit; both files verified byte-identical by sha256 afterwards):

mutation	result
record_new_screenshot guard → if false && …	RED
Snap#checkout_base_screenshot recording → if checked_out && false	RED
drop @checked_out_baselines.clear from reset	GREEN — a finding. Pinned by a new test ("SnapDiff.reset forgets which baselines were checked out"), which now reds

Gates: rake test:unit (698 runs) and rake test:canonical (587 runs) green locally and under CI=true; standardrb lib test clean.

Docs

docs/thread_safety.md and docs/reporters.md already stated that parallelize(with: :threads) is clean and that deprecation notices print once per fork worker. Updated here: the threads row now names the screenshot-name-uniqueness precondition, and the name-collision section says the vanished baseline is now an error rather than a silent pass.

Left on #217

• The fiber-model decision — out of scope for 2.0 (it changes semantics). It now has a concrete consumer: the summary-line counts must agree with the fiber-local registry under every execution mode.
• Deprecation notices once per fork worker — judged correct, not a bug: a forked child is a new process. Documented in the shared-state table rather than engineered around.
• fail_if_new_screenshot under record: :none — the victim of the same race raises the misleading "No existing screenshot found … git add" message. Loud, not silent, so not the bug this PR closes; the seam is in screenshot_matcher.rb, owned by another lane today.

Refs #217. Do not merge yet.

Summary by Sourcery

Prevent concurrent screenshot-name collisions from silently skipping baseline comparisons by distinguishing vanished checked-out baselines from genuinely new screenshots.

Bug Fixes:

• Raise an error when a baseline checked out for comparison disappears, preventing concurrent screenshot-name collisions from silently passing without verification.
• Preserve the existing behavior for screenshots with no previously committed baseline by recording them as new.

Enhancements:

• Track successful baseline checkouts per session and clear that state during reset.
• Document the screenshot-name uniqueness requirement for threaded parallel execution and the resulting vanished-baseline error.

Documentation:

• Update reporter and thread-safety documentation with the screenshot-name uniqueness requirement and baseline-loss failure behavior.

Tests:

• Add unit coverage for vanished baselines, legitimate new screenshots, session-state reset, and concurrent assertions sharing a screenshot name.

[sourcery-ai]

Sorry @pftg, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[sourcery-ai]

Reviewer's Guide

The PR closes the silent comparison-skip race by recording successful baseline checkouts in the session and raising when such a baseline disappears before comparison, with reset-safe tracking, race-focused regression tests, and updated thread-safety documentation.

Sequence diagram for detecting a disappeared screenshot baseline

sequenceDiagram
    participant Test
    participant Snap
    participant Git as SnapshotManager
    participant Session as AssertionRegistry
    participant Archive

    Test->>Snap: checkout_base_screenshot()
    Snap->>Git: checkout_file(path, base_path)
    Git-->>Snap: checked_out
    Snap->>Session: record_baseline_checkout(full_name)
    Test->>Session: need_to_compare?
    Archive->>Git: archive_baseline!
    Git-->>Session: baseline missing
    Session->>Session: record_new_screenshot(name)
    alt baseline was checked out
        Session-->>Test: raise SnapDiff::Error
    else no baseline was ever committed
        Session-->>Test: record new screenshot
    end

Loading

File-Level Changes

Change	Details	Files
Distinguish a vanished checked-out baseline from a legitimately missing baseline and fail loudly instead of recording a silently passing new screenshot.	Track successful baseline checkouts per session. Raise a descriptive error when a previously checked-out baseline is missing during comparison. Clear checkout tracking during session reset while preserving normal new-screenshot handling.	lib/snap_diff/screenshot_assertion.rb lib/snap_diff/snap.rb
Add regression coverage for deterministic disappearance, session cleanup, and the real concurrent name-collision race.	Verify vanished baselines raise and are not recorded as new. Verify names with no committed baseline remain valid new screenshots. Exercise concurrent same-name assertions and assert that none are silently skipped.	test/unit/baseline_disappeared_test.rb
Document the screenshot-name uniqueness requirement and the new loud failure mode for threaded execution.	Qualify threaded reporter guarantees with unique screenshot names. Explain that a disappeared baseline now raises and recommend unique names or non-concurrent execution.	docs/reporters.md docs/thread_safety.md

Possibly linked issues

• Fiber and thread safety: remaining decisions and hardening after the v2 core work #217: The PR directly implements the issue’s silently-wrong parallel execution guard by raising when a checked-out baseline disappears.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Warning

Review limit reached

Next included review available in 25 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e7734e27-ec0c-4e53-ae7c-62c6e087deca

📥 Commits

Reviewing files that changed from the base of the PR and between 31f68e6 and 804616e.

📒 Files selected for processing (5)

• docs/reporters.md
• docs/thread_safety.md
• lib/snap_diff/screenshot_assertion.rb
• lib/snap_diff/snap.rb
• test/unit/baseline_disappeared_test.rb

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}