Skip to content

fix(claude): leave out models the organization has restricted - #8652

Open
whoisaldo wants to merge 1 commit into
pingdotgg:mainfrom
whoisaldo:fix/claude-org-restricted-models
Open

fix(claude): leave out models the organization has restricted#8652
whoisaldo wants to merge 1 commit into
pingdotgg:mainfrom
whoisaldo:fix/claude-org-restricted-models

Conversation

@whoisaldo

@whoisaldo whoisaldo commented Aug 29, 2026

Copy link
Copy Markdown

What Changed

The Claude provider now leaves out catalog models the account's organization has not entitled, the same way it already leaves out models the installed Claude Code is too old for. The provider's status detail in Settings names them.

The capabilities probe reads the per-model entitlements Claude Code caches in its global config (modelAccessCache, the list its own /model menu is built from), so this costs no extra spawn and shares the probe's existing cache. Server and docs only: no contract, web, or mobile change.

One registry rule changes with it. A completed Claude health check now replaces the model list, the way a Codex refresh does since #9773, instead of retaining every model the previous snapshot listed. Without that, the boot-time snapshot's full catalog resurrected the withheld rows on the next merge. This also makes the existing version gate take effect: a model the installed CLI is too old for was likewise coming back from that snapshot.

Context and discussion in #8657.

Why

An organization can disallow individual Claude models. Claude Code does not fail those requests; it swaps in the org default and reports the swap partway through the turn. T3 Code never read that state, so on an account whose org restricts Fable 5, Fable 5 stayed selectable in every picker, choosing it ran Opus 5, and the thread still read "Claude Fable 5". Reproduced on a restricted account:

requested   = claude-fable-5
init.model  = claude-opus-5[1m]
answered    = claude-opus-5
notice      = [warning] Model "claude-fable-5" is restricted by your
              organization's settings. Using claude-opus-5[1m] instead.

The notice itself now surfaces since #9135 (SDK 0.3.260 declares informational). This PR removes the wrong pick before it happens. With the model absent from the list, a saved selection that pointed at it resolves to the provider default through the existing fallback, so the composer names the model that will actually answer instead of carrying a restricted name that would have substituted anyway.

Why leave them out rather than grey them out. An earlier revision of this PR disabled the rows with a reason, which needed a contract field and web and mobile rendering. Leaving them out matches what Claude Code's own /model menu does with entitlement-denied rows, matches how this provider already treats version-gated models, and keeps the change on the server. The withheld models are still visible: the status detail lists them, as the version upgrade message lists a model the CLI cannot run yet.

Why not the SDK. Checked against the CLI before settling on the cache:

  • The init models list is the CLI's curated picker with restricted rows already dropped or replaced by their fallback, so absence cannot be read as "restricted": claude-opus-4-8 is absent and runs normally.
  • The unavailable_models init field that would carry disabled rows is @internal, only populated for the VS Code entrypoint, and carries server-disabled rows rather than entitlement denials. Still true in SDK 0.3.260.
  • Probing set_model per catalog model would work today, but a health check that fires every few minutes should not lean on that request staying free of API traffic, which the interactive /model path already is not.

Safety. Entitlements apply only on the first-party and gateway backends, as in the CLI, so a cache left behind by an earlier claude.ai login cannot hide models a Bedrock, Vertex, or Foundry account can run. Custom models the user declared stay listed. Reading fails open otherwise: a missing, unreadable, or malformed config withholds nothing, entries are validated one at a time so one odd entry costs only itself, and only an explicit entitled: false counts. No capabilities probe means no entitlement list and nothing withheld. Config path resolution matches what the spawned CLI reads: the instance homePath exported as CLAUDE_CONFIG_DIR, then an ambient CLAUDE_CONFIG_DIR, then ~/.claude.json.

Not covered: the managed-settings availableModels allowlist, a separate mechanism with its own alias matching. A model blocked only by that allowlist still substitutes, with the notice from #9135.

UI Changes

Model picker on the org-restricted account, before and after. Note the ⌘1 jump shortcut in the before shot: the restricted row was a selectable keyboard target.

Model picker before, Claude Fable 5.1 listed and selectable on the restricted account

Model picker after, Claude Fable 5.1 and Claude Fable 5 no longer listed

Settings, the same provider's status detail, after:

Provider status detail reading Authenticated · Restricted by your organization: Claude Fable 5.1, Claude Fable 5.

Captured against a disposable .t3 seeded from live data, on a real org-restricted account, with the same build before and after. The personal account on the same machine keeps every model listed and no status note.

Verification

  • ClaudeEntitlements.test.ts, new: parsing including dated API ids, the config-dir and home precedences, and the fail-open paths, including that a relative config dir or home is never read.
  • End to end against the real CLI (2.1.261) and both real accounts on the machine, through the same probe and status check the server runs: the restricted account withholds both Fable models and reports the reason, the personal one lists all ten.
  • ProviderRegistry.test.ts: restricted catalog models are withheld and named in the status detail, a custom model stays listed, a Bedrock or unnamed backend ignores the cache, and the snapshot merge drops a model a completed check leaves out while a failed check keeps what was known.
  • ClaudeCapabilitiesProbe.test.ts: the probe surfaces entitlements, pointed at an isolated fixture so it never reads the developer's real config.
  • tsgo --noEmit clean on apps/server. vp lint and vp fmt clean on every changed file.

Checklist

  • This PR is small and focused
  • I explained what changed and why
  • I included before/after screenshots for any UI changes
  • I included a video for animation/interaction changes (not applicable)

Written by Claude Fable 5.1 in Claude Code.

Note

Filter out organization-restricted Claude models from catalog

  • Adds readClaudeRestrictedModels to parse .claude.json and collect model slugs with entitled: false, resolving config via CLAUDE_CONFIG_DIR or HOME; read/schema failures return an empty set
  • Normalizes dated Claude API model ids with toCatalogSlug so release-date suffixes do not block matching
  • probeClaudeCapabilities reads the entitlement set from the CLI environment and carries it in the capability snapshot
  • checkClaudeProviderStatus partitions first-party and gateway catalog models, removing restricted slugs and reporting their display names in provider status; Bedrock, Vertex, Foundry, and unknown backends keep the full catalog, and custom models always stay listed
  • shouldRetainMissingProviderModels now treats Claude like Antigravity/Codex, so a successful discovery replaces the cached inventory and omitted models disappear
  • Behavioral Change: ClaudeDriver.create now passes the provider-scoped FileSystem to the cached capability probe; first-party/gateway users with org restrictions will see fewer models in the picker
📊 Macroscope summarized 5cc82df. 5 files reviewed, 1 issue evaluated, 1 issue filtered, 0 comments posted

🗂️ Filtered Issues

apps/server/src/provider/Layers/ClaudeProvider.ts — 0 comments posted, 1 evaluated, 1 filtered
  • line 618: providerModelsFromSettings is called with only entitlement.entitled, then appends every configured custom model. A user can configure a custom model whose slug is a known restricted catalog model (for example claude-fable-5), which recreates a selectable entry that the preceding partition removed. Selecting it again lets the CLI silently substitute the organization default, so the chosen label does not match the model that runs. [ Already posted ]

@coderabbitai

coderabbitai Bot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 31532a77-a72b-4e32-b6d1-45de311b8ff5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Aug 29, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One consistency finding on the web changes: the new isUnavailable flag reaches every ProviderModelPicker call site, but the matching disabled treatment only exists in ChatView. Details inline.

Posted via Macroscope — UI Consistency

Comment thread apps/web/src/modelSelection.ts Outdated
@whoisaldo

Copy link
Copy Markdown
Author

Context for this change is in #8657, per CONTRIBUTING's note that non-trivial changes should start as a discussion.

Leaving this in draft until the mobile row has a before/after capture to match the web ones.

@whoisaldo
whoisaldo marked this pull request as ready for review August 29, 2026 14:21
@whoisaldo

Copy link
Copy Markdown
Author

Out of draft.

Correcting my earlier note about the mobile capture. I did attempt it. The Expo prebuild and pod install both completed, but the iOS build stops resolving packages because clerk-ios 1.3.6 needs Swift tools 6.2, which means Xcode 26. I had Xcode 16.4 and Swift 6.1.2. That matches the iOS 26.5 and iOS 27 simulators in your recent mobile PRs, so I assume this is expected and only affects me.

The mobile edit is 16 lines in ThreadSettingsSheet and modelOptions, covered by typecheck and unit tests. If you would rather it land separately, I am happy to pull it into its own PR and keep this one to server, web, and contracts.

@macroscopeapp

macroscopeapp Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces new entitlement parsing and automatically removes organization-restricted Claude models from the existing picker and provider snapshots. Because the behavior is user-facing and spans the capability probe, provider status, and snapshot registry, it merits human review.

You can add or adjust custom eligibility rules. Learn more.

Claude Code silently substitutes the org default for a model the account's
organization has not entitled, so an org that restricts Fable 5 left T3 Code
offering it as selectable, answering from Opus 5, and still labelling the
thread "Claude Fable 5". The picker had no way to know: the SDK's init model
list drops restricted rows rather than flagging them, so absence there cannot
be read as "restricted" (Opus 4.8 is absent from it and runs normally), and
the field that would carry them is internal to the VS Code extension.

Read the resolved per-model entitlements Claude Code caches in its global
config (`modelAccessCache`, the list its own `/model` menu is built from)
during the existing capabilities probe, and leave restricted catalog models
out of the provider snapshot, the way a model the installed CLI is too old
for is already left out. Clients then resolve a selection that pointed at a
restricted model to the provider default, so the label matches what answers.
The provider's status detail names the withheld models, as the version
upgrade message names a model the CLI cannot run yet.

Entitlements apply only on the first-party and gateway backends, as in the
CLI, so a cache left behind by an earlier claude.ai login cannot hide models
a Bedrock, Vertex, or Foundry account can run. Custom models stay listed.
Reading fails open otherwise: a missing, unreadable, or malformed cache
withholds nothing, one odd entry costs only itself, and only an explicit
`entitled: false` counts.

Written by Claude Fable 5.1 in Claude Code.
@whoisaldo
whoisaldo force-pushed the fix/claude-org-restricted-models branch from 8fab205 to 5cc82df Compare September 6, 2026 06:42
@whoisaldo whoisaldo changed the title fix(claude): grey out models the organization has restricted fix(claude): leave out models the organization has restricted Sep 6, 2026
@whoisaldo

Copy link
Copy Markdown
Author

Rebased on main and reduced to a server-only change.

Two things moved since the last push. #9135 bumped the SDK to 0.3.260, which declares informational, and handled its warning level, so the notice half of this PR is gone. And rather than disabling restricted rows with a reason, which needed a contract field plus web and mobile rendering, the provider now leaves restricted catalog models out of its snapshot, the way it already leaves out models the installed CLI is too old for, and names them in the status detail. That is also what Claude Code's own /model menu does with entitlement-denied rows. The earlier consistency finding about the three settings pickers is moot as a result: there is no row to disable.

Entitlements now only apply on the first-party and gateway backends, matching the CLI, so a stale cache cannot hide Bedrock or Vertex models.

One more change surfaced while verifying against the real account: the registry merge retained every model the previous snapshot listed for Claude, so the boot-time snapshot's full catalog brought the withheld rows straight back. A completed Claude check now replaces the list, as a Codex refresh does since #9773. That also makes the existing version gate take effect at runtime.

Captures are from a disposable .t3 seeded from live data on the same build before and after.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant