feat: an optional readiness block, and a run-level tally of selectors that never matched (#277)

docs/configuration.md

@@ -472,8 +472,6 @@ Capybara::Screenshot::Diff.area_size_limit = 42
 Sometimes you have expected change that you want to ignore.
 You can use the `skip_area` option with `[left, top, right, bottom]`
 or css selector like `'#footer'` or `'.container .skipped_element'` to the `screenshot` method to ignore an area.
-Be aware that if the selector is not in the page then the library will wait the default wait time for it to appear.
-Therefore, it is best to only use css selectors for skip_areas you know will be in the page:
 
 ```ruby
 test 'unstable area' do
@@ -482,6 +480,64 @@ test 'unstable area' do
 end
 ```
 
+**`skip_area` masks what is on the page at assertion time — it does not wait.**
+A selector is resolved against the DOM as it is when the screenshot is taken; one
+that matches nothing masks nothing, immediately. (Until 2.0 it blocked for
+`Capybara.default_max_wait_time` per unmatched selector — 5s each, on every
+screenshot. That wait is gone.)
+
+So content that arrives late — lazy-loaded images, JS-injected widgets, anything
+behind an unresolved fetch — has to be settled *before* the assertion, or its
+mask will be empty and the unstable region will be compared. Settle it in the
+readiness block described below.
+
+If a selector matched nothing in *every* screenshot of a run, the end-of-run
+summary names it:
+
+```
+[snap_diff] 1 selector never matched anything in this run: "artcile img". A selector that matches nothing masks nothing -- check for a typo or a stale selector.
+```
+
+That is a run-level fact on purpose. Per screenshot the gem cannot tell a typo
+from a page that legitimately has no images, so it says nothing; a selector that
+matched *somewhere* is doing its job and is never mentioned. Nothing is printed
+when every selector matched.
+
+### The readiness block
+
+`assert_matches_screenshot` and `capture_screenshot` (and the `screenshot` /
+`assert_no_screenshot_changes` wrappers) take an optional block. It runs once,
+after the enabled check and before the capture:
+
+```ruby
+test 'gallery' do
+  visit '/gallery'
+  assert_matches_screenshot 'gallery', skip_area: ['article img'] do
+    scroll_to :bottom
+    assert_text 'End of gallery'
+    scroll_to :top
+  end
+end
+```
+
+**Why a block rather than the line above it:** work in the block is skipped when
+screenshots are off. Both methods return immediately when
+`SnapDiff.config.enabled` (or `screenshot_enabled`) is false, so a
+`preload_all_images` written on the preceding line still pays for its browser
+round-trips — scroll, wait, scroll back — for a screenshot that is never taken.
+Inside the block it costs nothing, which is what makes turning visual tests off
+actually free.
+
+Errors raised in the block are yours and propagate unchanged. It is not a hook:
+there is no configuration-level equivalent, no after-block, and it runs once per
+assertion rather than once per stability retry.
+
+In RSpec, call `assert_matches_screenshot` directly rather than through the
+`match_screenshot` matcher — `expect(page).to match_screenshot('x') { ... }`
+binds the block by Ruby's `{}`/`do...end` precedence rather than by intent, so
+the matcher does not take one. In Cucumber the DSL is in the World, so step
+definitions pass a block the same way a Minitest test does.
+
 The arguments are `[left, top, right, bottom]` for the area you want to ignore.  You can also set this globally:
 
 ```ruby

lib/snap_diff/browser_helpers.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "snap_diff/region"
+require "snap_diff/reporting"
 
 module SnapDiff
   module BrowserHelpers
@@ -29,9 +30,16 @@ def self.window_size_is_wrong?(expected_window_size = nil)
         session.driver.browser.manage.window.size != ::Selenium::WebDriver::Dimension.new(*expected_window_size)
     end
 
+    # The one seam that knows, per selector, whether it found anything --
+    # `all_visible_regions_for` is called once per selector and the caller
+    # gets back one flattened list. The run-level tally is fed here (#277b)
+    # rather than in AreaCalculator for exactly that reason: by the time
+    # the regions are concatenated the attribution is gone.
     def self.bounds_for_css(*css_selectors)
       css_selectors.reduce([]) do |regions, selector|
-        regions.concat(all_visible_regions_for(selector))
+        found = all_visible_regions_for(selector)
+        SnapDiff::Reporting.record_selector_use(selector, matched: !found.empty?)
+        regions.concat(found)
       end
     end
 

lib/snap_diff/dsl.rb

@@ -64,13 +64,27 @@ def screenshot_group(name)
     # @option options [Numeric] :shift_distance_limit Maximum allowed shift distance for pixels.
     # @option options [Numeric] :area_size_limit Maximum allowed difference area size in pixels.
     # @option options [Symbol] :driver (:auto) The image processing driver to use (:auto, :chunky_png, :vips).
+    # @yield Optional readiness block: work that must happen before the page
+    #   is captured -- settling lazy-loaded images, `document.fonts.ready`,
+    #   waiting on a widget. Runs AFTER the `active?` guard and before the
+    #   capture, exactly once per assertion (not once per stability
+    #   attempt), and an error raised inside it propagates unchanged.
+    #
+    #   The point is not ergonomics -- while screenshots are on, the block
+    #   does nothing a line above the call could not. The point is what
+    #   happens when they are OFF: this method returns at the guard above,
+    #   so a preceding `preload_all_images` still costs its browser
+    #   round-trips while the block costs nothing. Readiness work belongs
+    #   inside the same switch as the capture it serves.
     # @return [Boolean] True if the screenshot was successfully captured and processed.
     # @raise [SnapDiff::ExpectationNotMet] If comparison fails and immediate validation is enabled.
     # @raise [SnapDiff::UnstableImage] If the image comparison is unstable.
     # @raise [SnapDiff::WindowSizeMismatchError] If the window size doesn't match expectations.
     def assert_matches_screenshot(name, skip_stack_frames: 0, **options)
       return false unless SnapDiff.config.active?
 
+      yield if block_given?
+
       # Get the full name with section and group information
       full_name = SnapDiff.session.screenshot_namer.full_name(name)
 
@@ -96,23 +110,29 @@ def assert_matches_screenshot(name, skip_stack_frames: 0, **options)
 
     # Convenience wrapper around {#assert_matches_screenshot} and {#capture_screenshot}.
     # @param compare [Boolean] When false, only captures the screenshot without comparing it to a baseline.
+    # @yield Forwarded to whichever of the two it delegates to. A delegator
+    #   that swallowed the block would give the user a readiness block that
+    #   silently never runs.
     # @see #assert_matches_screenshot
     # @see #capture_screenshot
-    def screenshot(name, skip_stack_frames: 0, compare: true, **options)
+    def screenshot(name, skip_stack_frames: 0, compare: true, **options, &readiness)
       if compare
-        assert_matches_screenshot(name, skip_stack_frames: skip_stack_frames + 1, **options)
+        assert_matches_screenshot(name, skip_stack_frames: skip_stack_frames + 1, **options, &readiness)
       else
-        capture_screenshot(name, **options)
+        capture_screenshot(name, **options, &readiness)
       end
     end
 
     # Captures a screenshot without comparing it to a baseline.
     # @param name [String] The base name of the screenshot, used to generate the filename.
     # @param options [Hash] Additional options for taking the screenshot. See {#assert_matches_screenshot}.
+    # @yield Optional readiness block. See {#assert_matches_screenshot}.
     # @return [Boolean] True if the screenshot was successfully captured.
     def capture_screenshot(name, **options)
       return false unless SnapDiff.config.active?
 
+      yield if block_given?
+
       full_name = SnapDiff.session.screenshot_namer.full_name(name)
       SnapDiff::ScreenshotMatcher.new(full_name, options).capture
 
@@ -122,8 +142,8 @@ def capture_screenshot(name, **options)
     # Asserts the current page has no visual changes from the baseline.
     # Override in your base test class to add project-specific behavior
     # (e.g., waiting for Turbo, default skip areas).
-    def assert_no_screenshot_changes(name, skip_stack_frames: 0, **opts)
-      assert_matches_screenshot(name, skip_stack_frames: skip_stack_frames + 1, **opts)
+    def assert_no_screenshot_changes(name, skip_stack_frames: 0, **opts, &readiness)
+      assert_matches_screenshot(name, skip_stack_frames: skip_stack_frames + 1, **opts, &readiness)
     end
   end
 end

lib/snap_diff/reporting.rb

@@ -19,6 +19,8 @@ module Reporting
     @mutex = Mutex.new
     @missing_baselines = Set.new
     @rerecorded_baselines = Set.new
+    @matched_selectors = Set.new
+    @unmatched_selectors = Set.new
     @verified = 0
     @changed = 0
 
@@ -57,6 +59,21 @@ def missing_baselines_count
         @mutex.synchronize { @missing_baselines.size }
       end
 
+      # Remembers whether a CSS selector (`skip_area`, `crop`) found
+      # anything the one time it was resolved. Fed at the point of use, so
+      # a selector that was configured but never reached cannot be named.
+      #
+      # Two sets rather than a counter: the question the summary answers is
+      # "did this selector match ANYWHERE in the run", and under
+      # fork-parallel the hits and the misses arrive from different
+      # processes. Subtracting at read time is the only shape that survives
+      # that merge (#266).
+      def record_selector_use(selector, matched:)
+        @mutex.synchronize do
+          (matched ? @matched_selectors : @unmatched_selectors) << selector
+        end
+      end
+
       # @api private
       # Per-test isolation for this gem's own suite: everything {finalize!}
       # reports, cleared in one call. One surface rather than one reset per
@@ -65,6 +82,8 @@ def reset_run_totals!
         @mutex.synchronize do
           @missing_baselines.clear
           @rerecorded_baselines.clear
+          @matched_selectors.clear
+          @unmatched_selectors.clear
           @verified = 0
           @changed = 0
         end
@@ -199,6 +218,10 @@ def finalize!
         if (msg = rerecorded_baselines_summary)
           $stdout.puts msg
         end
+
+        if (msg = never_matched_selectors_summary)
+          $stdout.puts msg
+        end
       end
 
       # --- fork-parallel reports (issue #258) ---------------------------
@@ -249,6 +272,11 @@ def dump_parallel_fragment
         payload = {
           "missing_baselines" => @mutex.synchronize { @missing_baselines.to_a },
           "rerecorded_baselines" => @mutex.synchronize { @rerecorded_baselines.to_a },
+          # Both halves, not the subtraction: `img` may match in this worker
+          # and miss in the next, and only the parent that merged every
+          # fragment can tell whether it matched anywhere in the run.
+          "matched_selectors" => @mutex.synchronize { @matched_selectors.to_a },
+          "unmatched_selectors" => @mutex.synchronize { @unmatched_selectors.to_a },
           "verified" => @verified,
           "changed" => @changed,
           "reporters" => @mutex.synchronize { @reporters.dup }
@@ -283,6 +311,8 @@ def merge_parallel_fragments!
           @mutex.synchronize do
             payload["missing_baselines"].each { |name| @missing_baselines << name }
             payload.fetch("rerecorded_baselines", []).each { |name| @rerecorded_baselines << name }
+            payload.fetch("matched_selectors", []).each { |selector| @matched_selectors << selector }
+            payload.fetch("unmatched_selectors", []).each { |selector| @unmatched_selectors << selector }
             @verified += payload.fetch("verified", 0)
             @changed += payload.fetch("changed", 0)
           end
@@ -327,6 +357,32 @@ def rerecorded_baselines_summary
         "[snap_diff] record: :all re-recorded #{label} WITHOUT comparing: #{names.join(", ")}. " \
           "Review the result before committing -- an unintended change is accepted just as silently."
       end
+
+      # The selectors that matched nothing in EVERY screenshot of the run.
+      #
+      # Since #272 a `skip_area` selector is resolved without waiting: it
+      # masks what is on the page at assertion time, and one that matches
+      # nothing produces an empty mask -- the unstable region is compared
+      # and the test flakes, silently. #275 declined to warn per screenshot
+      # because the gem cannot tell a typo from a legitimately image-less
+      # page, and the legitimate case would fire on every screenshot.
+      #
+      # A run-level tally has no such problem. A selector that matched
+      # SOMEWHERE is doing its job and is never mentioned; one that matched
+      # NOWHERE, all run, is a typo or a stale selector with high
+      # probability. Silent when the set is empty, on purpose: a line that
+      # prints on every run is a line users learn to skip.
+      #
+      # @return [String, nil] nil when every selector used matched somewhere
+      def never_matched_selectors_summary
+        names = @mutex.synchronize { (@unmatched_selectors - @matched_selectors).to_a }
+        return if names.empty?
+
+        label = (names.size == 1) ? "1 selector" : "#{names.size} selectors"
+        "[snap_diff] #{label} never matched anything in this run: " \
+          "#{names.map(&:inspect).join(", ")}. " \
+          "A selector that matches nothing masks nothing -- check for a typo or a stale selector."
+      end
     end
   end
 end

test/integration/browser_screenshot_test.rb

@@ -236,6 +236,23 @@ def test_screenshot_selected_element
     assert_equal 1, SnapDiff::BrowserHelpers.bounds_for_css("img").size
   end
 
+  # The other half of the #272 trade, against a REAL browser (#277b). The
+  # implicit wait was accidentally giving late-loading elements time to
+  # appear; without it an unmatched selector yields an empty mask and says
+  # nothing. Per screenshot the gem cannot tell a typo from a legitimately
+  # image-less page -- but across a whole run it can, and this is the seam
+  # where the answer is known per selector.
+  test "resolving selectors feeds the run-level never-matched tally" do
+    visit "/"
+
+    SnapDiff::BrowserHelpers.bounds_for_css("img", "picture")
+
+    summary = SnapDiff::Reporting.never_matched_selectors_summary
+
+    assert_includes summary, "picture"
+    refute_includes summary, "img", "a selector that matched was accused of never matching"
+  end
+
   test "rect_for for multiple elements returns first visible element" do
     visit "/index.html"