Skip to content

Add Safari build coverage to pull request CI - #1023

Merged
PeterDaveHello merged 2 commits into
masterfrom
addSafariMacCi
Jul 21, 2026
Merged

Add Safari build coverage to pull request CI#1023
PeterDaveHello merged 2 commits into
masterfrom
addSafariMacCi

Conversation

@PeterDaveHello

@PeterDaveHello PeterDaveHello commented Jul 21, 2026

Copy link
Copy Markdown
Member

Summary

  • run the existing Safari packaging workflow on macos-14 for pull
    requests and master updates that can affect release output
  • include Safari and tagged-release workflow changes in PR CI triggers
  • verify the generated app signature and DMG integrity without release
    signing credentials
  • make Safari packaging fail fast and keep repeated builds isolated from
    Firefox release artifacts
  • install Darwin-only appdmg@0.6.6 from the committed lockfile
  • remove the obsolete empty Xcode project patch

Why

Safari packaging previously had no macOS coverage in pull request CI.
The existing build script also installed appdmg dynamically and
continued after failed intermediate commands, making its CI signal
non-reproducible and susceptible to false success.

The new job matches the current release runner and exercises the same
locked, fail-fast packaging path without release signing credentials.
The tagged-release workflow only changes toolchain setup ordering and
removes version-substitution steps that became no-ops; publishing and
release signing behavior are unchanged.

Validation

  • based on latest master (89cf0a4)
  • npm run pretty
  • npm run lint
  • npm test (893 tests passed)
  • npm run build
  • required Chromium and Firefox build artifacts verified
  • Safari-only manifest patch verified against generated output while
    firefox.zip retains the ChatGPTBox name
  • shellcheck safari/build.sh
  • bash -n safari/build.sh
  • workflow YAML parsing
  • git diff --check
  • local CodeRabbit, Claude, and OpenCode review loops found no remaining
    actionable issues
  • GitHub Actions tests passed on Linux
  • GitHub Actions safari-build passed on macOS, including the Xcode
    archive, ad-hoc app signature, and DMG integrity validation

Manual browser smoke testing was skipped because no extension runtime
files changed.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • ✅ Review completed - (🔄 Check again to review again)
📝 Walkthrough

Walkthrough

The pull request updates Safari build scripting and its optional DMG dependency, configures Python and Xcode in tagged releases, and adds macOS CI coverage that builds and verifies Safari app and DMG artifacts.

Changes

Safari build and release integration

Layer / File(s) Summary
Safari build pipeline
safari/build.sh, safari/project.pre.patch, package.json
The build uses strict Bash behavior, applies the manifest metadata patch, exports the Safari app, and generates the DMG through the pinned optional appdmg dependency.
Safari CI triggers and validation
.github/workflows/pr-tests.yml
Safari changes trigger a macOS build that configures Node, Python, and Xcode, then validates the app bundle, codesign result, and DMG. The coverage badge job also waits for the Safari build.
Tagged release toolchain
.github/workflows/tagged-release.yml
Tagged releases configure Python and Xcode for all workflow events and no longer rewrite Safari patch files with the release version.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant SafariBuild
  participant SafariConverter
  participant Xcodebuild
  participant ArtifactChecks
  GitHubActions->>SafariBuild: run npm run build:safari
  SafariBuild->>SafariConverter: convert generated extension
  SafariConverter->>Xcodebuild: archive and export Safari app
  SafariBuild-->>GitHubActions: create Safari app and DMG
  GitHubActions->>ArtifactChecks: verify bundle, codesign, and DMG
  ArtifactChecks-->>GitHubActions: pass or fail build
Loading

Possibly related PRs

Suggested labels: dependencies, github_actions

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding Safari build coverage to pull request CI.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch addSafariMacCi

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Add Safari macOS build + artifact verification to PR CI

⚙️ Configuration changes ✨ Enhancement 🕐 10-20 Minutes

Grey Divider

AI Description

• Expand PR/push path filters to include Safari packaging inputs
• Add a macOS-15 CI job running build:safari with Xcode 16.2
• Verify the produced .app bundle signature and safari.dmg readability
Diagram

graph TD
  E{{"PR / Push"}} --> F["Path filters (src/, safari/, build.mjs, tests/, package.json)"] --> W["CI workflow: pr-tests.yml"]
  W --> J1["Job: tests"]
  W --> J2["Job: safari-build (macOS-15)"] --> V["Verify artifacts (codesign, hdiutil)"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Split Safari packaging into a dedicated workflow
  • ➕ Keeps pr-tests.yml focused on unit/integration tests
  • ➕ Allows separate concurrency limits and clearer ownership
  • ➕ Easier to disable/iterate without touching core CI
  • ➖ More workflow files to maintain
  • ➖ Harder to see all PR-required checks in one place
  • ➖ May duplicate shared setup steps
2. Use a reusable workflow shared by PR CI and release packaging
  • ➕ Avoids drift between PR validation and release packaging
  • ➕ Centralizes Xcode/node setup and artifact verification logic
  • ➖ Refactor overhead now (more YAML indirection)
  • ➖ Harder for casual contributors to follow without context
3. Run Safari build via a matrix job (linux + macOS)
  • ➕ Single job definition with OS-specific branches
  • ➕ Consistent environment setup patterns
  • ➖ More conditional logic inside one job
  • ➖ Couples unrelated responsibilities (unit tests vs packaging)

Recommendation: The current approach (a separate macOS-only safari-build job inside the existing PR workflow) is a good balance: it adds missing macOS/Xcode coverage with minimal refactoring, keeps permissions read-only, and validates concrete packaging artifacts. Consider extracting to a reusable workflow later if release and PR packaging logic start diverging.

Files changed (1) +39 / -0

Other (1) +39 / -0
pr-tests.ymlAdd macOS Safari packaging job and expand path filters +39/-0

Add macOS Safari packaging job and expand path filters

• Adds safari/** to PR and master push path filters so Safari packaging-related changes trigger CI. Introduces a new macos-15 safari-build job that installs Node/Python/Xcode, runs npm run build:safari, and validates the generated .app bundle and DMG without requiring release signing credentials.

.github/workflows/pr-tests.yml

@qodo-code-review

qodo-code-review Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Mutable action tags 🐞 Bug ⛨ Security ⭐ New
Description
The workflows use mutable GitHub Action tags (e.g., @v7, @v6, @v1) rather than immutable
commit SHAs, so CI/release behavior can change without any repository change if those tags move.
This weakens build reproducibility and increases CI/CD supply-chain risk, particularly for the
Safari packaging path that produces distributable artifacts.
Code

.github/workflows/pr-tests.yml[R56-67]

+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 22
+      - uses: actions/setup-python@v6
+        with:
+          python-version: '3.10'
+      - uses: maxim-lobanov/setup-xcode@v1
+        with:
+          xcode-version: 16.2
Evidence
The new/updated workflows explicitly reference actions via major-version tags rather than commit
SHAs, which are mutable pointers resolved at runtime.

.github/workflows/pr-tests.yml[39-67]
.github/workflows/tagged-release.yml[35-49]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Workflows reference actions by mutable tags (e.g. `actions/checkout@v7`, `maxim-lobanov/setup-xcode@v1`). Tags can be moved, which can change CI/release behavior without any change in this repo.

## Issue Context
This PR adds/updates workflow steps for Safari build coverage and release setup. Pinning actions to a full commit SHA makes CI runs reproducible and reduces supply-chain exposure.

## Fix Focus Areas
- .github/workflows/pr-tests.yml[40-67]
- .github/workflows/tagged-release.yml[36-48]

## Implementation notes
- Replace each `uses: owner/action@vN` with `uses: owner/action@<full_sha>`.
- Optionally keep the human-readable version as a comment, e.g. `# v7`.
- Apply consistently to the actions introduced/modified by this PR (and ideally across all workflows for consistency).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Empty patch allowed silently 🐞 Bug ☼ Reliability
Description
safari/build.sh applies safari/project.patch with --allow-empty, so if the patch is
unintentionally emptied/truncated the build will still succeed and silently skip any intended Xcode
project modifications. In the current branch, safari/project.patch contains only file headers and no
hunks, so the apply step is effectively a no-op by design or accident.
Code

safari/build.sh[10]

+git apply --allow-empty safari/project.patch
Evidence
The build script explicitly allows an empty patch, and the current patch file shows no hunks (only
file headers), meaning the apply step can succeed while making no changes.

safari/build.sh[5-16]
safari/project.patch[1-2]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`safari/build.sh` uses `git apply --allow-empty safari/project.patch`, which permits an empty patch file to succeed. That removes a failure signal if `project.patch` is accidentally reduced to an empty/no-op patch, potentially skipping required Xcode project adjustments.

### Issue Context
`project.patch` is currently effectively empty (headers only). If that’s intentional, it’s safer to remove the apply step or add an explicit assertion so future accidental truncation is caught.

### Fix Focus Areas
- safari/build.sh[6-16]
- safari/project.patch[1-2]

### Suggested fix
Choose one:
1) If no patching is needed: delete `safari/project.patch` and remove the `git apply ... safari/project.patch` line.
2) If patching may be needed later: keep the file, but add a guard such as `grep -q '^@@' safari/project.patch` (or similar) and fail with a clear error if it’s unexpectedly empty, and only use `--allow-empty` when you explicitly expect emptiness (e.g., gated by a variable/flag).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Release workflow not CI-triggered ✓ Resolved 🐞 Bug ☼ Reliability
Description
Because pr-tests.yml is path-filtered and does not include .github/workflows/tagged-release.yml, PRs
that only change the release workflow will not run PR CI, leaving release/Safari packaging changes
unvalidated until a tag is pushed. This becomes more relevant in this PR because tagged-release.yml
was modified (Python/Xcode setup ordering and Safari build invocation).
Code

.github/workflows/pr-tests.yml[R9-14]

    paths:
      - "src/**"
+      - "safari/**"
      - "build.mjs"
      - "tests/**"
      - "package.json"
Evidence
The PR-test workflow is explicitly limited to a set of paths and currently excludes the release
workflow file, while the release workflow contains the Safari build/release steps that can change
independently.

.github/workflows/pr-tests.yml[4-29]
.github/workflows/tagged-release.yml[31-94]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`pr-tests.yml` uses `on.pull_request.paths` and `on.push.paths` filters, but they don’t include `.github/workflows/tagged-release.yml`. As a result, PRs that only adjust the release workflow won’t run PR CI, so release/Safari packaging workflow changes can merge without validation.

### Issue Context
This PR changes `tagged-release.yml`, but PR CI triggering is governed by `pr-tests.yml` path filters.

### Fix Focus Areas
- .github/workflows/pr-tests.yml[4-29]

### Suggested fix
Add `.github/workflows/tagged-release.yml` (and any other release-related workflows you expect to validate) to both the `pull_request.paths` and `push.paths` lists in `pr-tests.yml`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (3)
4. Safari build not idempotent 🐞 Bug ☼ Reliability
Description
safari/build.sh applies safari/project.pre.patch to the tracked src/manifest.v2.json but never
reverses it, leaving the working tree dirty after success or failure. A subsequent `npm run
build:safari will typically fail immediately because git apply safari/project.pre.patch` can’t
re-apply an already-applied patch.
Code

safari/build.sh[R5-10]

git apply safari/project.pre.patch
npm run build
xcrun safari-web-extension-converter ./build/firefox \
 --project-location ./build/safari --app-name "Fission - ChatBox" \
 --bundle-identifier dev.josStorer.chatGPTBox --force --no-prompt --no-open
-git apply safari/project.patch
+git apply --allow-empty safari/project.patch
Evidence
The build script applies patches to tracked files but contains no cleanup or reversal logic; the
pre-patch explicitly targets src/manifest.v2.json, so leaving it applied makes the repo dirty and
prevents reapplying the same patch on subsequent runs.

safari/build.sh[1-16]
safari/project.pre.patch[1-7]
src/manifest.v2.json[1-6]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`safari/build.sh` applies `safari/project.pre.patch` (modifies tracked files) and does not restore the repository state afterward. With `set -euo pipefail`, any mid-script failure exits early and also leaves the patch applied, so rerunning `npm run build:safari` commonly fails at the initial `git apply`.

### Issue Context
- `safari/build.sh` runs `git apply safari/project.pre.patch` before building.
- `safari/project.pre.patch` changes `src/manifest.v2.json` (tracked file), so the checkout becomes dirty and the patch becomes non-reapplicable.

### Fix Focus Areas
- safari/build.sh[1-16]

### Suggested fix
Add a cleanup mechanism that restores the working tree even on failure, e.g.:
- Track whether `project.pre.patch` was applied successfully.
- Use `trap '...' EXIT` to revert changes:
 - `git checkout -- src/manifest.v2.json` (or `git apply -R safari/project.pre.patch` if you want exact reversal semantics)
 - optionally `git apply -R safari/project.patch || true` if you keep applying it

This keeps local runs repeatable and prevents reruns from failing due to already-applied patches.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Unpinned appdmg install ✓ Resolved 🐞 Bug ⛨ Security
Description
The new safari-build job runs npm run build:safari, and that script installs appdmg
dynamically during the build, outside the repo’s locked npm ci dependency set, making CI results
non-reproducible and prone to breakage when upstream appdmg changes. This also increases
supply-chain exposure because every PR build can fetch a new, unreviewed package version from the
registry.
Code

.github/workflows/pr-tests.yml[67]

+      - run: npm run build:safari
Evidence
The workflow change introduces execution of npm run build:safari in PR CI, and the referenced
script installs appdmg at build time rather than via the lockfile-driven npm ci step, so the
job’s dependency set can drift over time.

.github/workflows/pr-tests.yml[53-83]
safari/build.sh[11-13]
package.json[6-12]
package.json[75-112]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Safari CI job executes `npm run build:safari`, which currently performs a runtime `npm install -D appdmg` without pinning/locking it in `package-lock.json`. This makes CI builds dependent on whatever `appdmg` resolves to at build time.

### Issue Context
To keep PR CI deterministic and reduce supply-chain risk, dependencies used in CI should be installed via `npm ci` from the committed lockfile, not added dynamically during the build.

### Fix Focus Areas
- Add `appdmg` to `devDependencies` (pin a specific version) and commit the updated `package-lock.json`.
- Remove the runtime `npm install -D appdmg` from the Safari build script.

- .github/workflows/pr-tests.yml[53-83]
- safari/build.sh[1-13]
- package.json[6-25]
- package.json[75-112]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Build not fail-fast ✓ Resolved 🐞 Bug ☼ Reliability
Description
npm run build:safari executes safari/build.sh, which does not enable fail-fast behavior (no `set
-e`), so intermediate failures (e.g., patch application or Xcode steps) can be ignored and later
commands may still run. This reduces the reliability of the new CI signal because the workflow’s
post-step verification only checks for existence/validity of the resulting .app and .dmg, not
that every build step succeeded as intended.
Code

.github/workflows/pr-tests.yml[R67-82]

+      - run: npm run build:safari
+      - name: Verify Safari build artifacts
+        shell: bash
+        run: |
+          set -eo pipefail
+          shopt -s nullglob
+          apps=(build/*.app)
+
+          if [ "${#apps[@]}" -ne 1 ]; then
+            echo "Expected exactly one Safari app bundle, found ${#apps[@]}"
+            exit 1
+          fi
+
+          codesign --verify --deep --strict --verbose=2 "${apps[0]}"
+          test -s build/safari.dmg
+          hdiutil verify build/safari.dmg
Evidence
The workflow newly runs npm run build:safari, and the script it calls is a plain list of commands
with no fail-fast configuration; the workflow’s follow-up checks validate artifact
presence/signature but don’t guarantee earlier steps (like patching) succeeded.

.github/workflows/pr-tests.yml[67-82]
safari/build.sh[1-13]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new CI job relies on `safari/build.sh` to produce correct artifacts, but the script is a sequence of commands without fail-fast settings. A failed intermediate command can be missed, weakening CI’s ability to catch regressions.

### Issue Context
The workflow’s verification step validates some outputs (`codesign`, `hdiutil verify`), but it does not guarantee earlier steps (like patch application) were successful.

### Fix Focus Areas
- Add `set -euo pipefail` (or equivalent explicit status checks) at the top of `safari/build.sh`.
- Ensure critical steps (`git apply`, `xcrun`, `xcodebuild`, `appdmg`) stop the script on failure.

- .github/workflows/pr-tests.yml[68-82]
- safari/build.sh[1-13]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

7. macOS runner mismatch ✓ Resolved 🐞 Bug ☼ Reliability
Description
PR CI runs Safari packaging on macos-15 while the release workflow runs on macos-14, so PR
validation may not reflect the actual release environment and can miss runner-image-specific
packaging/signing issues. This is a coverage gap between PR validation and release builds.
Code

.github/workflows/pr-tests.yml[R48-49]

+    runs-on: macos-15
+    timeout-minutes: 20
Evidence
The PR workflow explicitly sets macos-15 for the new Safari job, while the existing release
workflow uses macos-14, demonstrating the environment mismatch.

.github/workflows/pr-tests.yml[47-50]
.github/workflows/tagged-release.yml[31-34]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new PR Safari job uses a different macOS runner version than the release workflow. That can lead to PR CI passing while release CI fails (or vice versa) due to OS image differences.

### Issue Context
Even with the same Xcode version configured, the underlying runner image can differ in system tooling and defaults.

### Fix Focus Areas
- Consider switching the PR job to `macos-14` to match release, or update the release workflow to `macos-15`, or run a small matrix to cover both.

- .github/workflows/pr-tests.yml[47-50]
- .github/workflows/tagged-release.yml[31-34]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Previous review results

Review updated until commit 1bc18b8

Results up to commit 2f2e92a ⚖️ Balanced


🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Remediation recommended
1. Build not fail-fast ✓ Resolved 🐞 Bug ☼ Reliability
Description
npm run build:safari executes safari/build.sh, which does not enable fail-fast behavior (no `set
-e`), so intermediate failures (e.g., patch application or Xcode steps) can be ignored and later
commands may still run. This reduces the reliability of the new CI signal because the workflow’s
post-step verification only checks for existence/validity of the resulting .app and .dmg, not
that every build step succeeded as intended.
Code

.github/workflows/pr-tests.yml[R67-82]

+      - run: npm run build:safari
+      - name: Verify Safari build artifacts
+        shell: bash
+        run: |
+          set -eo pipefail
+          shopt -s nullglob
+          apps=(build/*.app)
+
+          if [ "${#apps[@]}" -ne 1 ]; then
+            echo "Expected exactly one Safari app bundle, found ${#apps[@]}"
+            exit 1
+          fi
+
+          codesign --verify --deep --strict --verbose=2 "${apps[0]}"
+          test -s build/safari.dmg
+          hdiutil verify build/safari.dmg
Evidence
The workflow newly runs npm run build:safari, and the script it calls is a plain list of commands
with no fail-fast configuration; the workflow’s follow-up checks validate artifact
presence/signature but don’t guarantee earlier steps (like patching) succeeded.

.github/workflows/pr-tests.yml[67-82]
safari/build.sh[1-13]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new CI job relies on `safari/build.sh` to produce correct artifacts, but the script is a sequence of commands without fail-fast settings. A failed intermediate command can be missed, weakening CI’s ability to catch regressions.

### Issue Context
The workflow’s verification step validates some outputs (`codesign`, `hdiutil verify`), but it does not guarantee earlier steps (like patch application) were successful.

### Fix Focus Areas
- Add `set -euo pipefail` (or equivalent explicit status checks) at the top of `safari/build.sh`.
- Ensure critical steps (`git apply`, `xcrun`, `xcodebuild`, `appdmg`) stop the script on failure.

- .github/workflows/pr-tests.yml[68-82]
- safari/build.sh[1-13]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Unpinned appdmg install ✓ Resolved 🐞 Bug ⛨ Security
Description
The new safari-build job runs npm run build:safari, and that script installs appdmg
dynamically during the build, outside the repo’s locked npm ci dependency set, making CI results
non-reproducible and prone to breakage when upstream appdmg changes. This also increases
supply-chain exposure because every PR build can fetch a new, unreviewed package version from the
registry.
Code

.github/workflows/pr-tests.yml[67]

+      - run: npm run build:safari
Evidence
The workflow change introduces execution of npm run build:safari in PR CI, and the referenced
script installs appdmg at build time rather than via the lockfile-driven npm ci step, so the
job’s dependency set can drift over time.

.github/workflows/pr-tests.yml[53-83]
safari/build.sh[11-13]
package.json[6-12]
package.json[75-112]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Safari CI job executes `npm run build:safari`, which currently performs a runtime `npm install -D appdmg` without pinning/locking it in `package-lock.json`. This makes CI builds dependent on whatever `appdmg` resolves to at build time.

### Issue Context
To keep PR CI deterministic and reduce supply-chain risk, dependencies used in CI should be installed via `npm ci` from the committed lockfile, not added dynamically during the build.

### Fix Focus Areas
- Add `appdmg` to `devDependencies` (pin a specific version) and commit the updated `package-lock.json`.
- Remove the runtime `npm install -D appdmg` from the Safari build script.

- .github/workflows/pr-tests.yml[53-83]
- safari/build.sh[1-13]
- package.json[6-25]
- package.json[75-112]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational
3. macOS runner mismatch ✓ Resolved 🐞 Bug ☼ Reliability
Description
PR CI runs Safari packaging on macos-15 while the release workflow runs on macos-14, so PR
validation may not reflect the actual release environment and can miss runner-image-specific
packaging/signing issues. This is a coverage gap between PR validation and release builds.
Code

.github/workflows/pr-tests.yml[R48-49]

+    runs-on: macos-15
+    timeout-minutes: 20
Evidence
The PR workflow explicitly sets macos-15 for the new Safari job, while the existing release
workflow uses macos-14, demonstrating the environment mismatch.

.github/workflows/pr-tests.yml[47-50]
.github/workflows/tagged-release.yml[31-34]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new PR Safari job uses a different macOS runner version than the release workflow. That can lead to PR CI passing while release CI fails (or vice versa) due to OS image differences.

### Issue Context
Even with the same Xcode version configured, the underlying runner image can differ in system tooling and defaults.

### Fix Focus Areas
- Consider switching the PR job to `macos-14` to match release, or update the release workflow to `macos-15`, or run a small matrix to cover both.

- .github/workflows/pr-tests.yml[47-50]
- .github/workflows/tagged-release.yml[31-34]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit 677219c ⚖️ Balanced


🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Remediation recommended
1. Safari build not idempotent 🐞 Bug ☼ Reliability
Description
safari/build.sh applies safari/project.pre.patch to the tracked src/manifest.v2.json but never
reverses it, leaving the working tree dirty after success or failure. A subsequent `npm run
build:safari will typically fail immediately because git apply safari/project.pre.patch` can’t
re-apply an already-applied patch.
Code

safari/build.sh[R5-10]

git apply safari/project.pre.patch
npm run build
xcrun safari-web-extension-converter ./build/firefox \
 --project-location ./build/safari --app-name "Fission - ChatBox" \
 --bundle-identifier dev.josStorer.chatGPTBox --force --no-prompt --no-open
-git apply safari/project.patch
+git apply --allow-empty safari/project.patch
Evidence
The build script applies patches to tracked files but contains no cleanup or reversal logic; the
pre-patch explicitly targets src/manifest.v2.json, so leaving it applied makes the repo dirty and
prevents reapplying the same patch on subsequent runs.

safari/build.sh[1-16]
safari/project.pre.patch[1-7]
src/manifest.v2.json[1-6]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`safari/build.sh` applies `safari/project.pre.patch` (modifies tracked files) and does not restore the repository state afterward. With `set -euo pipefail`, any mid-script failure exits early and also leaves the patch applied, so rerunning `npm run build:safari` commonly fails at the initial `git apply`.

### Issue Context
- `safari/build.sh` runs `git apply safari/project.pre.patch` before building.
- `safari/project.pre.patch` changes `src/manifest.v2.json` (tracked file), so the checkout becomes dirty and the patch becomes non-reapplicable.

### Fix Focus Areas
- safari/build.sh[1-16]

### Suggested fix
Add a cleanup mechanism that restores the working tree even on failure, e.g.:
- Track whether `project.pre.patch` was applied successfully.
- Use `trap '...' EXIT` to revert changes:
 - `git checkout -- src/manifest.v2.json` (or `git apply -R safari/project.pre.patch` if you want exact reversal semantics)
 - optionally `git apply -R safari/project.patch || true` if you keep applying it

This keeps local runs repeatable and prevents reruns from failing due to already-applied patches.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit c266ff0 ⚖️ Balanced


🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Remediation recommended
1. Empty patch allowed silently 🐞 Bug ☼ Reliability
Description
safari/build.sh applies safari/project.patch with --allow-empty, so if the patch is
unintentionally emptied/truncated the build will still succeed and silently skip any intended Xcode
project modifications. In the current branch, safari/project.patch contains only file headers and no
hunks, so the apply step is effectively a no-op by design or accident.
Code

safari/build.sh[10]

+git apply --allow-empty safari/project.patch
Evidence
The build script explicitly allows an empty patch, and the current patch file shows no hunks (only
file headers), meaning the apply step can succeed while making no changes.

safari/build.sh[5-16]
safari/project.patch[1-2]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`safari/build.sh` uses `git apply --allow-empty safari/project.patch`, which permits an empty patch file to succeed. That removes a failure signal if `project.patch` is accidentally reduced to an empty/no-op patch, potentially skipping required Xcode project adjustments.

### Issue Context
`project.patch` is currently effectively empty (headers only). If that’s intentional, it’s safer to remove the apply step or add an explicit assertion so future accidental truncation is caught.

### Fix Focus Areas
- safari/build.sh[6-16]
- safari/project.patch[1-2]

### Suggested fix
Choose one:
1) If no patching is needed: delete `safari/project.patch` and remove the `git apply ... safari/project.patch` line.
2) If patching may be needed later: keep the file, but add a guard such as `grep -q '^@@' safari/project.patch` (or similar) and fail with a clear error if it’s unexpectedly empty, and only use `--allow-empty` when you explicitly expect emptiness (e.g., gated by a variable/flag).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Release workflow not CI-triggered ✓ Resolved 🐞 Bug ☼ Reliability
Description
Because pr-tests.yml is path-filtered and does not include .github/workflows/tagged-release.yml, PRs
that only change the release workflow will not run PR CI, leaving release/Safari packaging changes
unvalidated until a tag is pushed. This becomes more relevant in this PR because tagged-release.yml
was modified (Python/Xcode setup ordering and Safari build invocation).
Code

.github/workflows/pr-tests.yml[R9-14]

    paths:
      - "src/**"
+      - "safari/**"
      - "build.mjs"
      - "tests/**"
      - "package.json"
Evidence
The PR-test workflow is explicitly limited to a set of paths and currently excludes the release
workflow file, while the release workflow contains the Safari build/release steps that can change
independently.

.github/workflows/pr-tests.yml[4-29]
.github/workflows/tagged-release.yml[31-94]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`pr-tests.yml` uses `on.pull_request.paths` and `on.push.paths` filters, but they don’t include `.github/workflows/tagged-release.yml`. As a result, PRs that only adjust the release workflow won’t run PR CI, so release/Safari packaging workflow changes can merge without validation.

### Issue Context
This PR changes `tagged-release.yml`, but PR CI triggering is governed by `pr-tests.yml` path filters.

### Fix Focus Areas
- .github/workflows/pr-tests.yml[4-29]

### Suggested fix
Add `.github/workflows/tagged-release.yml` (and any other release-related workflows you expect to validate) to both the `pull_request.paths` and `push.paths` lists in `pr-tests.yml`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Qodo Logo

Comment thread .github/workflows/pr-tests.yml
Comment thread .github/workflows/pr-tests.yml
Comment thread .github/workflows/pr-tests.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Safari packaging validation to the PR CI workflow so macOS/Xcode-specific regressions surface earlier, without involving release signing credentials.

Changes:

  • Extend PR/push path filters to include safari/**.
  • Add a macos-15 safari-build job that runs npm run build:safari.
  • Add artifact verification for exactly one exported .app bundle plus DMG integrity checks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 16.2
- run: npm run build:safari

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

future works

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
safari/build.sh (1)

15-16: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider using npx appdmg for direct script execution.

Because appdmg was moved to package.json's optionalDependencies, its executable will reside in node_modules/.bin. If this script is executed directly (./safari/build.sh) instead of via an npm script (e.g., npm run build:safari), the appdmg command will not be found.

Using npx --no-install appdmg ensures that the local binary is reliably resolved regardless of how the script is executed.

♻️ Proposed fix
 rm -f ./build/safari.dmg
-appdmg ./safari/appdmg.json ./build/safari.dmg
+npx --no-install appdmg ./safari/appdmg.json ./build/safari.dmg
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@safari/build.sh` around lines 15 - 16, Update the appdmg invocation in the
Safari build script to use npx --no-install, ensuring the locally installed
optional dependency is resolved when the script runs directly. Keep the existing
input and output arguments unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/tagged-release.yml:
- Around line 43-46: Remove the event-name condition from the
actions/setup-python@v6 step in the tagged release workflow so Python 3.10 is
configured for every workflow trigger, including manual runs, before the
existing npm ci step executes.

In `@safari/build.sh`:
- Around line 1-16: Make the patch application flow in the build script
rerunnable: update the handling around git apply safari/project.pre.patch and
git apply --allow-empty safari/project.patch so already-applied patches are
safely reverted, skipped, or otherwise guarded without triggering set -e
failures. Preserve the existing build, conversion, archive, export, and
packaging sequence.

---

Nitpick comments:
In `@safari/build.sh`:
- Around line 15-16: Update the appdmg invocation in the Safari build script to
use npx --no-install, ensuring the locally installed optional dependency is
resolved when the script runs directly. Keep the existing input and output
arguments unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3b7bf8ea-ad0c-4675-b781-bfe5ade2ba1d

📥 Commits

Reviewing files that changed from the base of the PR and between 2f2e92a and 677219c.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (5)
  • .github/workflows/pr-tests.yml
  • .github/workflows/tagged-release.yml
  • package.json
  • safari/build.sh
  • safari/project.pre.patch
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/pr-tests.yml

Comment thread .github/workflows/tagged-release.yml
Comment thread safari/build.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread safari/build.sh Outdated
Comment thread .github/workflows/tagged-release.yml
Comment thread safari/build.sh Outdated
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 677219c

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread .github/workflows/pr-tests.yml Outdated
Comment thread safari/build.sh Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/tagged-release.yml:
- Around line 43-48: Update the release job’s macOS runner from macos-14 to a
supported runner, using macos-15-intel if the workflow requires x86_64. Verify
that the existing Xcode setup via maxim-lobanov/setup-xcode and Python 3.10
configuration for appdmg remain compatible with the selected runner.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 867d0d88-70a5-4e20-862f-160f00aeaf4b

📥 Commits

Reviewing files that changed from the base of the PR and between 677219c and c266ff0.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (5)
  • .github/workflows/pr-tests.yml
  • .github/workflows/tagged-release.yml
  • package.json
  • safari/build.sh
  • safari/project.pre.patch
🚧 Files skipped from review as they are similar to previous changes (3)
  • safari/project.pre.patch
  • safari/build.sh
  • .github/workflows/pr-tests.yml

Comment on lines +43 to +48
- uses: actions/setup-python@v6
with:
python-version: '3.10' # for appdmg
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 16.2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

sed -n '1,120p' .github/workflows/tagged-release.yml

Repository: ChatGPTBox-dev/chatGPTBox

Length of output: 3914


🌐 Web query:

GitHub Actions runner images macos-14 deprecation support end arm64 macos-14-large macos-15-intel current documentation

💡 Result:

The macOS 14-based GitHub Actions runner images are currently in the deprecation process and will reach their end of support on November 2, 2026 [1][2]. Key details regarding this deprecation: - Deprecation process: The deprecation of macOS 14 and macOS 14 arm64 images began on July 6, 2026 [1][2]. - End of Support: These images will be fully unsupported for both GitHub Actions and Azure DevOps starting November 2, 2026 [1][2]. - Affected labels: Workflows utilizing the macos-14, macos-14-large, or macos-14-xlarge labels are affected [1]. These jobs will be terminated with an error after the retirement date [1]. Recommended Migration: Users are advised to update their workflows to use more recent image labels [1]. Current alternatives include: - For general use: macos-latest, macos-15, or macos-26 [1]. - For large runners: macos-latest-large, macos-15-large, or macos-26-large [1]. - For xlarge runners: macos-latest-xlarge, macos-15-xlarge, or macos-26-xlarge [1]. Note on macos-15-intel: While the user inquired about macos-15-intel, current documentation identifies support for macOS 15 and macOS 26 environments, with specific Intel-based runner options available as macos-15-intel and macos-26-intel [3]. Users should monitor the official runner-images repository for specific image availability and version updates [4][5].

Citations:


Migrate this release job off macos-14. macos-14 is in deprecation and will stop being supported on Nov 2, 2026, so this release workflow will eventually fail. Move to a supported runner (for example macos-15-intel if x86_64 is required) and recheck Xcode 16.2 and appdmg compatibility.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/tagged-release.yml around lines 43 - 48, Update the
release job’s macOS runner from macos-14 to a supported runner, using
macos-15-intel if the workflow requires x86_64. Verify that the existing Xcode
setup via maxim-lobanov/setup-xcode and Python 3.10 configuration for appdmg
remain compatible with the selected runner.

Source: MCP tools

Comment thread .github/workflows/pr-tests.yml
Comment thread safari/build.sh Outdated
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit c266ff0

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment on lines +43 to +48
- uses: actions/setup-python@v6
with:
python-version: '3.10' # for appdmg
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 16.2
Comment on lines +49 to +54
safari-build:
runs-on: macos-14
timeout-minutes: 20
permissions:
contents: read

contents: read

steps:
- uses: actions/checkout@v7

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Mutable action tags 🐞 Bug ⛨ Security

The workflows use mutable GitHub Action tags (e.g., @v7, @v6, @v1) rather than immutable
commit SHAs, so CI/release behavior can change without any repository change if those tags move.
This weakens build reproducibility and increases CI/CD supply-chain risk, particularly for the
Safari packaging path that produces distributable artifacts.
Agent Prompt
## Issue description
Workflows reference actions by mutable tags (e.g. `actions/checkout@v7`, `maxim-lobanov/setup-xcode@v1`). Tags can be moved, which can change CI/release behavior without any change in this repo.

## Issue Context
This PR adds/updates workflow steps for Safari build coverage and release setup. Pinning actions to a full commit SHA makes CI runs reproducible and reduces supply-chain exposure.

## Fix Focus Areas
- .github/workflows/pr-tests.yml[40-67]
- .github/workflows/tagged-release.yml[36-48]

## Implementation notes
- Replace each `uses: owner/action@vN` with `uses: owner/action@<full_sha>`.
- Optionally keep the human-readable version as a comment, e.g. `# v7`.
- Apply consistently to the actions introduced/modified by this PR (and ideally across all workflows for consistency).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 8016c44

Make Safari packaging fail on intermediate errors and install appdmg
from the lockfile after configuring the macOS toolchain.

Apply the Safari-only name change to generated Firefox output so
repeated builds leave tracked manifests and Firefox archives intact.

Remove the obsolete empty Xcode project patch.
Run Safari packaging on macOS for pull requests and master updates
that can affect release output.

Verify the app bundle signature and disk image without release
credentials, and require Safari validation before badge updates.
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 1bc18b8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.

@PeterDaveHello
PeterDaveHello merged commit e16be6d into master Jul 21, 2026
5 checks passed
@PeterDaveHello
PeterDaveHello deleted the addSafariMacCi branch July 21, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants