Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,31 @@ bin/rails test:system # now green — HEAD holds the new baseline
Reviewing the change is what the pull request is for — the updated `.png` shows up as an image
diff next to the code that caused it.

### Accepting many at once

One failure at a time is fine for one screenshot. After a redesign that changed forty, set the
**record mode** to `:all` and re-record them in a single run:

```ruby
# test_helper.rb
SnapDiff.config.record = ENV["ACCEPT_SCREENSHOTS"] ? :all : :once
```

```bash
ACCEPT_SCREENSHOTS=1 bin/rails test:system
# [snap_diff] record: :all re-recorded 40 screenshots WITHOUT comparing: ...

git status # forty modified baselines, and nothing else
git add doc/screenshots/
git commit -m "chore: re-record baselines after the checkout redesign"
```

`:all` writes every capture straight to its baseline path and compares nothing, so `git status`
is the review surface — look at the images before you commit. It **refuses to run under CI**,
where there would be nobody to look. The other modes are `:once` (the default) and `:none`
(strict — a missing baseline always fails); see
[Record modes](docs/configuration.md#record-modes--accepting-changes).

## Web UI for Reviewing Screenshot Changes

Add one line to get an interactive dashboard for reviewing all screenshot differences:
Expand Down
45 changes: 41 additions & 4 deletions docs/UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Version 2.0 introduces a new canonical namespace (`SnapDiff`) for cleaner, more discoverable code. The public DSL remains unchanged — your existing `screenshot` and `assert_matches_screenshot` calls work without modification. This guide covers the optional migration path for settings and the new namespace.

**Status:** 2.0 is the **transitional** release — the v1 API and the canonical `SnapDiff` API both work. **2.1 removes** everything 2.0 warns about (the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction). There is no 3.0. Migrating on 2.0 is optional; doing it before 2.1 is not.
**Status:** 2.0 is the **transitional** release — the v1 API and the canonical `SnapDiff` API both work. **2.1 removes** everything 2.0 warns about (the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction, and the `fail_if_new` / `pending_if_new` / `fail_on_difference` booleans that `record` replaces). There is no 3.0. Migrating on 2.0 is optional; doing it before 2.1 is not.

**Estimated upgrade time:** 5–15 minutes (most users need only the Gemfile pin)

Expand Down Expand Up @@ -114,7 +114,7 @@ The most commonly-used settings and how to update them:
| `save_path` | `Capybara::Screenshot.save_path = "doc/screenshots"` | `SnapDiff.config.save_path = "doc/screenshots"` | Where baseline screenshots are stored |
| `window_size` | `Capybara::Screenshot.window_size = [1280, 1024]` | `SnapDiff.config.window_size = [1280, 1024]` | Browser viewport size for consistent screenshots |

**All 27 settings** from both legacy namespaces are available via `SnapDiff.config.<attr_name>` — see the [Configuration Reference](configuration.md) for the full list. One rename to note: `Capybara::Screenshot.enabled` becomes `SnapDiff.config.screenshot_enabled` (it would otherwise collide with `Capybara::Screenshot::Diff.enabled`, which keeps the bare `enabled` name).
**All 28 settings** from both legacy namespaces are available via `SnapDiff.config.<attr_name>` — see the [Configuration Reference](configuration.md) for the full list. One rename to note: `Capybara::Screenshot.enabled` becomes `SnapDiff.config.screenshot_enabled` (it would otherwise collide with `Capybara::Screenshot::Diff.enabled`, which keeps the bare `enabled` name).

---

Expand Down Expand Up @@ -172,9 +172,10 @@ This means you can migrate your codebase incrementally **now**, before opting in

### Deprecation Warnings

v2.0 emits four different things, and it is worth knowing which is which. The first two are
v2.0 emits five different things, and it is worth knowing which is which. The first two are
about the old namespaces; the third is about the driver features 2.1 removes; the fourth is
about options that never did anything.
about options that never did anything; the fifth is about the new-screenshot booleans that
`record` replaces.

Everything 2.1 removes warns in 2.0, and every warning names 2.1. Nothing you can still write
in 2.0 does nothing quietly — if a setting is on its way out, or was never read at all, you
Expand Down Expand Up @@ -293,6 +294,42 @@ It applies to every route into a comparison — `screenshot`, `assert_matches_sc
`median_filter_window_size`, `perceptual_threshold`, `screenshot_format`,
`shift_distance_limit`, `skip_area`, `stability_time_limit`, `tolerance` and `wait`.

#### 5. The new-screenshot booleans — superseded by `record`, removed in 2.1

`fail_if_new`, `pending_if_new` and `fail_on_difference` each answered part of "what happens when
there is no baseline, or when there is a difference" — and none of them named the action people
actually want, which is *accept this change*. 2.0 adds the verb:

```ruby
SnapDiff.config.record = :once # default. Record a screenshot that has no baseline.
SnapDiff.config.record = :none # strict. A missing baseline always fails.
SnapDiff.config.record = :all # re-record everything. THE BULK-ACCEPT MODE.
```

| Old | New | Note |
|-----|-----|------|
| `fail_if_new = true` | `record = :none` | the mode means the same thing on CI and off it |
| `fail_if_new = false` | `record = :once` | |
| `pending_if_new = true` | `record = :none`, or `:once` | `:none` fails with the `git add` command attached; `:once` records and lists it in the end-of-run summary |
| `fail_on_difference = false` | `record = :all` | to *accept* the new rendering rather than ignore the difference |

All three keep working for the whole 2.x line (2.0 deletes nothing) and each warns once per
process, from the point you set it:

```
[snap_diff deprecation] `fail_if_new` is REMOVED in 2.1: the record modes replace it. `SnapDiff.config.record = :none` is `fail_if_new = true`, `= :once` is `fail_if_new = false` -- and unlike the boolean, a mode means the same thing on CI and off it. See docs/configuration.md. Silence with `SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. (shown once per process) (called from /app/test/test_helper.rb:9)
```

**Nothing changes if you set none of them.** With no `record` line, `record` reads back as `:none`
under CI and `:once` off it — exactly what `fail_if_new` already did, sniff and all. The
missing-baseline default is deliberately unchanged; `:none` is how you opt into strictness
explicitly. **Precedence:** an explicitly set mode outranks `fail_if_new`, and `fail_if_new`
decides only when no mode was set — the same rule `fail_if_new` itself has over the `CI` sniff.

`record = :all` **refuses to run under CI**: it accepts every rendering by design, so left in a
committed config file it would be a build that compares nothing and passes forever. See
[Record modes](configuration.md#record-modes--accepting-changes).

#### Silent by design

Some legacy names never warn individually, and that is deliberate — the migration notice above is
Expand Down
8 changes: 4 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,13 +234,13 @@ The two legacy views are organized into two namespaces:

**`Capybara::Screenshot::Diff`** — comparison settings:
- `driver`, `tolerance`, `color_distance_limit`, `perceptual_threshold`, `shift_distance_limit`
- `area_size_limit`, `skip_area`, `fail_if_new`, `fail_on_difference`, `delayed`
- `area_size_limit`, `skip_area`, `record`, `fail_if_new`, `fail_on_difference`, `delayed`

The canonical way in is `SnapDiff.configure { |config| ... }` (all 27 settings flat on one object). `SnapDiff.start` and `Capybara::Screenshot::Diff.configure` are the two-holder block shape over the same storage — since ADR-008 step 7b, `Diff.configure` forwards to `SnapDiff.start` rather than the other way round.
The canonical way in is `SnapDiff.configure { |config| ... }` (all 28 settings flat on one object). `SnapDiff.start` and `Capybara::Screenshot::Diff.configure` are the two-holder block shape over the same storage — since ADR-008 step 7b, `Diff.configure` forwards to `SnapDiff.start` rather than the other way round.

`Config` also owns the derived values that used to live on the legacy modules: `active?` (ex `Capybara::Screenshot.active?`), `screenshot_area` / `screenshot_area_abs`, and `default_options` (ex `Capybara::Screenshot::Diff.default_options`, the option hash handed to `SnapDiff::Comparison`). The legacy module methods one-line forward here.

**Default timing contract:** every *stored* default is evaluated once, in `Config#initialize`, which runs at require time of `config.rb` — the same load moment the old `mattr_accessor` default blocks evaluated at. `root` (from `Rails.root`) must never become a lazy read-time default. Two values are deliberately live: `default_options[:wait]`, a method-body read of `Capybara.default_max_wait_time`, and `fail_if_new`, whose reader falls back to `ENV["CI"]` whenever nothing explicit was set — an explicit setting outranks the environment, so the sniff cannot be frozen into storage.
**Default timing contract:** every *stored* default is evaluated once, in `Config#initialize`, which runs at require time of `config.rb` — the same load moment the old `mattr_accessor` default blocks evaluated at. `root` (from `Rails.root`) must never become a lazy read-time default. Two values are deliberately live: `default_options[:wait]`, a method-body read of `Capybara.default_max_wait_time`, and `fail_if_new`, whose reader falls back to `ENV["CI"]` whenever nothing explicit was set — an explicit setting outranks the environment, so the sniff cannot be frozen into storage. `record` is the third: its reader falls back to `fail_if_new` (`:none` when it is on, `:once` when it is off), which is what makes the record modes a rename of the existing behaviour rather than a change to it — the matcher branches on the mode alone and a suite with no `record` line takes exactly the branches it always took.

## File Layout

Expand All @@ -249,7 +249,7 @@ lib/
snap_diff.rb # SnapDiff module: compare/start/configure/config
snap_diff/ # Canonical implementation (v2)
dsl.rb # screenshot(), screenshot_group(), etc.
config.rb # SnapDiff::Config — THE storage for all 27 settings
config.rb # SnapDiff::Config — THE storage for all 28 settings
errors.rb # Error / ExpectationNotMet / UnstableImage / WindowSizeMismatchError
region.rb # SnapDiff::Region — bounding box (+ eager top-level ::Region alias)
deprecation.rb # Warn-once-per-constant machinery
Expand Down
26 changes: 17 additions & 9 deletions docs/ci-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,15 +253,23 @@ jobs:

</details>

> **`CI: ""` is what makes this job able to record a *new* baseline.** `fail_if_new`
> defaults to `true` whenever `ENV["CI"]` is set and non-empty, and the check runs
> **before** the capture ([`screenshot_matcher.rb`](https://github.com/snap-diff/snap_diff-capybara/blob/master/lib/snap_diff/screenshot_matcher.rb)
> — `check_base_screenshot` precedes `capture_screenshot`). So on a stock GitHub Actions
> runner a screenshot with no committed baseline raises `No existing screenshot found for
> …` and **nothing is written to disk** — the commit step then finds nothing to commit,
> whatever the error message suggests. Clearing `CI` for this one step (or setting
> `SnapDiff.config.fail_if_new = false`) lets both new and changed baselines land.
> *Changed* baselines are rewritten either way; only new ones need this.
> **Set the record mode, or clear `CI`, to let this job record a *new* baseline.** With
> nothing set, a missing baseline fails whenever `ENV["CI"]` is set and non-empty — so on a
> stock GitHub Actions runner a screenshot with no committed baseline raises `No existing
> screenshot found for …`. `SnapDiff.config.record = :once`
> (or clearing `CI` for this one step, or the older `SnapDiff.config.fail_if_new = false`)
> records new baselines instead. *Changed* baselines are rewritten either way; only new ones
> need this. See [Record modes](configuration.md#record-modes--accepting-changes).
>
> The screenshot itself **is** written before the raise ([`screenshot_matcher.rb`](https://github.com/snap-diff/snap_diff-capybara/blob/master/lib/snap_diff/screenshot_matcher.rb)
> — `capture_screenshot` precedes `fail_if_new_screenshot`), so the `git add` the message names
> is a command you can actually run. But the raise still fails the test it happened in, and a red
> job usually never reaches the commit step — which is why a recording job sets the mode rather
> than relying on the files being there.
>
> **`record = :all` is not the mode for this job** — it refuses to run under CI, because it
> would accept every *changed* rendering unreviewed as well. `:once` records what is new and
> keeps comparing everything that already has a baseline.

**How it works:**
1. Go to Actions → "Update Screenshot Baselines" → "Run workflow"
Expand Down
Loading
Loading