fix(claude): leave out models the organization has restricted - #8652
fix(claude): leave out models the organization has restricted#8652whoisaldo wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
There was a problem hiding this comment.
One consistency finding on the web changes: the new isUnavailable flag reaches every ProviderModelPicker call site, but the matching disabled treatment only exists in ChatView. Details inline.
Posted via Macroscope — UI Consistency
|
Context for this change is in #8657, per CONTRIBUTING's note that non-trivial changes should start as a discussion. Leaving this in draft until the mobile row has a before/after capture to match the web ones. |
|
Out of draft. Correcting my earlier note about the mobile capture. I did attempt it. The Expo prebuild and The mobile edit is 16 lines in |
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR introduces new entitlement parsing and automatically removes organization-restricted Claude models from the existing picker and provider snapshots. Because the behavior is user-facing and spans the capability probe, provider status, and snapshot registry, it merits human review. You can add or adjust custom eligibility rules. Learn more. |
Claude Code silently substitutes the org default for a model the account's organization has not entitled, so an org that restricts Fable 5 left T3 Code offering it as selectable, answering from Opus 5, and still labelling the thread "Claude Fable 5". The picker had no way to know: the SDK's init model list drops restricted rows rather than flagging them, so absence there cannot be read as "restricted" (Opus 4.8 is absent from it and runs normally), and the field that would carry them is internal to the VS Code extension. Read the resolved per-model entitlements Claude Code caches in its global config (`modelAccessCache`, the list its own `/model` menu is built from) during the existing capabilities probe, and leave restricted catalog models out of the provider snapshot, the way a model the installed CLI is too old for is already left out. Clients then resolve a selection that pointed at a restricted model to the provider default, so the label matches what answers. The provider's status detail names the withheld models, as the version upgrade message names a model the CLI cannot run yet. Entitlements apply only on the first-party and gateway backends, as in the CLI, so a cache left behind by an earlier claude.ai login cannot hide models a Bedrock, Vertex, or Foundry account can run. Custom models stay listed. Reading fails open otherwise: a missing, unreadable, or malformed cache withholds nothing, one odd entry costs only itself, and only an explicit `entitled: false` counts. Written by Claude Fable 5.1 in Claude Code.
8fab205 to
5cc82df
Compare
|
Rebased on main and reduced to a server-only change. Two things moved since the last push. #9135 bumped the SDK to 0.3.260, which declares Entitlements now only apply on the first-party and gateway backends, matching the CLI, so a stale cache cannot hide Bedrock or Vertex models. One more change surfaced while verifying against the real account: the registry merge retained every model the previous snapshot listed for Claude, so the boot-time snapshot's full catalog brought the withheld rows straight back. A completed Claude check now replaces the list, as a Codex refresh does since #9773. That also makes the existing version gate take effect at runtime. Captures are from a disposable |
What Changed
The Claude provider now leaves out catalog models the account's organization has not entitled, the same way it already leaves out models the installed Claude Code is too old for. The provider's status detail in Settings names them.
The capabilities probe reads the per-model entitlements Claude Code caches in its global config (
modelAccessCache, the list its own/modelmenu is built from), so this costs no extra spawn and shares the probe's existing cache. Server and docs only: no contract, web, or mobile change.One registry rule changes with it. A completed Claude health check now replaces the model list, the way a Codex refresh does since #9773, instead of retaining every model the previous snapshot listed. Without that, the boot-time snapshot's full catalog resurrected the withheld rows on the next merge. This also makes the existing version gate take effect: a model the installed CLI is too old for was likewise coming back from that snapshot.
Context and discussion in #8657.
Why
An organization can disallow individual Claude models. Claude Code does not fail those requests; it swaps in the org default and reports the swap partway through the turn. T3 Code never read that state, so on an account whose org restricts Fable 5, Fable 5 stayed selectable in every picker, choosing it ran Opus 5, and the thread still read "Claude Fable 5". Reproduced on a restricted account:
The notice itself now surfaces since #9135 (SDK 0.3.260 declares
informational). This PR removes the wrong pick before it happens. With the model absent from the list, a saved selection that pointed at it resolves to the provider default through the existing fallback, so the composer names the model that will actually answer instead of carrying a restricted name that would have substituted anyway.Why leave them out rather than grey them out. An earlier revision of this PR disabled the rows with a reason, which needed a contract field and web and mobile rendering. Leaving them out matches what Claude Code's own
/modelmenu does with entitlement-denied rows, matches how this provider already treats version-gated models, and keeps the change on the server. The withheld models are still visible: the status detail lists them, as the version upgrade message lists a model the CLI cannot run yet.Why not the SDK. Checked against the CLI before settling on the cache:
modelslist is the CLI's curated picker with restricted rows already dropped or replaced by their fallback, so absence cannot be read as "restricted":claude-opus-4-8is absent and runs normally.unavailable_modelsinit field that would carry disabled rows is@internal, only populated for the VS Code entrypoint, and carries server-disabled rows rather than entitlement denials. Still true in SDK 0.3.260.set_modelper catalog model would work today, but a health check that fires every few minutes should not lean on that request staying free of API traffic, which the interactive/modelpath already is not.Safety. Entitlements apply only on the first-party and gateway backends, as in the CLI, so a cache left behind by an earlier claude.ai login cannot hide models a Bedrock, Vertex, or Foundry account can run. Custom models the user declared stay listed. Reading fails open otherwise: a missing, unreadable, or malformed config withholds nothing, entries are validated one at a time so one odd entry costs only itself, and only an explicit
entitled: falsecounts. No capabilities probe means no entitlement list and nothing withheld. Config path resolution matches what the spawned CLI reads: the instancehomePathexported asCLAUDE_CONFIG_DIR, then an ambientCLAUDE_CONFIG_DIR, then~/.claude.json.Not covered: the managed-settings
availableModelsallowlist, a separate mechanism with its own alias matching. A model blocked only by that allowlist still substitutes, with the notice from #9135.UI Changes
Model picker on the org-restricted account, before and after. Note the
⌘1jump shortcut in the before shot: the restricted row was a selectable keyboard target.Settings, the same provider's status detail, after:
Captured against a disposable
.t3seeded from live data, on a real org-restricted account, with the same build before and after. The personal account on the same machine keeps every model listed and no status note.Verification
ClaudeEntitlements.test.ts, new: parsing including dated API ids, the config-dir and home precedences, and the fail-open paths, including that a relative config dir or home is never read.ProviderRegistry.test.ts: restricted catalog models are withheld and named in the status detail, a custom model stays listed, a Bedrock or unnamed backend ignores the cache, and the snapshot merge drops a model a completed check leaves out while a failed check keeps what was known.ClaudeCapabilitiesProbe.test.ts: the probe surfaces entitlements, pointed at an isolated fixture so it never reads the developer's real config.tsgo --noEmitclean onapps/server.vp lintandvp fmtclean on every changed file.Checklist
Written by Claude Fable 5.1 in Claude Code.
Note
Filter out organization-restricted Claude models from catalog
readClaudeRestrictedModelsto parse.claude.jsonand collect model slugs withentitled: false, resolving config viaCLAUDE_CONFIG_DIRorHOME; read/schema failures return an empty settoCatalogSlugso release-date suffixes do not block matchingprobeClaudeCapabilitiesreads the entitlement set from the CLI environment and carries it in the capability snapshotcheckClaudeProviderStatuspartitions first-party and gateway catalog models, removing restricted slugs and reporting their display names in provider status; Bedrock, Vertex, Foundry, and unknown backends keep the full catalog, and custom models always stay listedshouldRetainMissingProviderModelsnow treats Claude like Antigravity/Codex, so a successful discovery replaces the cached inventory and omitted models disappearClaudeDriver.createnow passes the provider-scopedFileSystemto the cached capability probe; first-party/gateway users with org restrictions will see fewer models in the picker📊 Macroscope summarized 5cc82df. 5 files reviewed, 1 issue evaluated, 1 issue filtered, 0 comments posted
🗂️ Filtered Issues
apps/server/src/provider/Layers/ClaudeProvider.ts — 0 comments posted, 1 evaluated, 1 filtered
providerModelsFromSettingsis called with onlyentitlement.entitled, then appends every configured custom model. A user can configure a custom model whose slug is a known restricted catalog model (for exampleclaude-fable-5), which recreates a selectable entry that the preceding partition removed. Selecting it again lets the CLI silently substitute the organization default, so the chosen label does not match the model that runs. [ Already posted ]