Skip to content

Rule screens show what a description became, labelled as what OpenBot checks - #1572

Open
guidovizoso wants to merge 9 commits into
mainfrom
guido/rules-say-deterministic
Open

guidovizoso wants to merge 9 commits into
mainfrom
guido/rules-say-deterministic

Conversation

@guidovizoso

@guidovizoso guidovizoso commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Admins asked whether the rules they write on the governance screens are applied by a model each time. For most of them they are not, but the screens didn't say so. When an admin describes a rule in words, a model drafts it once. What gets saved and checked is the draft, not the words. A sentence saying so is not enough. So each describe-in-words flow now shows the exact thing the words became, before the admin saves, and labels it as what OpenBot checks.

No behaviour changes. Every screen shows data the draft response already returns. No new server fields.

What each describe flow shows after a draft

Flow What's shown Label text
Boundaries, Describe the rule (Add rule, and a rule's page) The CEL in the Rule field. On Add rule the field now sits right under the description when you describe a rule, before Name and What it stops. "Your model drafted this from your description. This is the rule OpenBot checks before every action, exactly as written."
Permissions, Describe who can do what The exact switches: permission, Everybody, Admins, Members or group, and On, Off, Grant or Inherit. They now sit above the reading. "Your model drafted these from your description. They are the switches OpenBot checks, exactly as set. Remove any you don't want before you save."
Computers, Network, Describe the allowlist The entries in the Allowlist field, and the mode above it "Your model drafted this list and the mode above from your description. OpenBot checks every connection against them, exactly as written."
An app's page for one Bot, Describe what this Bot can do New list, "Actions [Bot] may use": each action the model chose, by its exact name. If the app doesn't offer an action, the reason shows under its name. "Your model chose these from your description. Once you save, [Bot] may use exactly these [app] actions and no others. OpenBot checks every call against this list."
Approvals, Describe team rules The rules (tool, kind of action, target, Bot, behaviour). They now sit above the reading. "Your model drafted these from your description. OpenBot matches each action against them exactly as written." Under any rule set to Take action if pre-approved: "The reviewer model still checks that the person asked for this exact action."

In every flow, the "second reading" summary stays as the plain-language explanation, under the exact artifact.

The describe fields no longer say "OpenBot checks the X you save, not your words." The label on the artifact says that now. The Boundaries field says it drafts "the rule in CEL".

What each screen enforces

I checked the server code (as it was before #1530 moved it out of this repo) to see how each saved setting is enforced.

Screen How the saved setting is checked Model at check time? Change
Boundaries list, Add rule, a rule's page CEL expressions evaluated by cel-js (computer/policy.ts), deny before allow, fails closed No "checked exactly as written, no model decides". The drafted CEL is labelled as the rule OpenBot checks
Permissions, and Describe who can do what Switch lookups over organization, role and group rows (admin/capabilities.ts, pure) No Same. The drafted switches are labelled as what OpenBot checks
Computers, Network section and its policy dialog Allowlist matcher on the computer (agent-computer/src/egress.ts), pushed by computer/policy-network.ts No Same. The drafted list and mode are labelled as what OpenBot checks
An app's page for one Bot ("Describe what this Bot can do") Per-action grants, a stored list No The chosen actions are listed and labelled as the only ones the Bot may use
Approvals, team rules, and Describe team rules Which actions a rule covers is a glob match (approvals/policy.ts). But a "pre-approved" rule asks the reviewer model whether the person asked for this exact action, and with auto-review on the reviewer can make an allowed action ask Partly The drafted rules are labelled as matched exactly as written. A pre-approved rule says the reviewer model decides. I did not add "no model decides" here
Approvals, Guidance for auto-review Text the reviewer model reads with every action it checks Yes None
Approvals, Require auto-review A reviewer model checks each action Yes None

Page sentences kept from the first change

  • Boundaries: "Each rule is checked exactly as written, the same way every time. No model decides."
  • Add rule: "It is checked exactly as written, the same way every time. No model decides."
  • A rule's page, Rule section: "Written in CEL. It is checked exactly as written before every action a Bot takes, the same way every time. No model decides."
  • Permissions, and Describe who can do what: "Permissions are checked exactly as set, the same way every time. No model decides." (and "Saved permissions ..." on the describe page)
  • Computers, Network: "Each connection is checked against the policy exactly as written, the same way every time. No model decides."
  • The "Describe it in words" rows on Boundaries, Permissions and Approvals say the model "drafts".

Tests

Each test renders a returned draft and checks for the exact artifact, its label, and that the reading comes after it.

  • app/tests/boundaries-describe.test.tsx: on Add rule, the drafted CEL is labelled and sits between the description and Name. On a rule's page it is labelled only once something has been drafted.
  • app/tests/admin-permissions-describe.test.tsx (new): the drafted switch (Local computer access, Members, Off) is labelled and comes before the reading.
  • app/tests/admin-network-describe.test.tsx: the drafted entries are labelled with the reading under them. A list typed by hand is not called a draft.
  • app/tests/bot-app-describe-grants.test.tsx (new): the chosen actions are listed and labelled. An action the app doesn't offer shows its reason.
  • app/tests/approval-rules-describe.test.tsx: the rules are labelled and come before the reading. Only a pre-approved rule mentions the reviewer model.

All 9 new or changed assertions fail on the previous head (5c2b2dd) and pass here. These suites pass: Boundaries, admin, approvals, Bot app, grants and plugins (286 tests across 45 files). biome lint, biome format and the app typecheck pass too.

Checked in a browser

I ran the server image with its own database, and the app from this branch on 127.0.0.1, with a real OpenAI model drafting every flow. I ran all five flows end to end, and also turned on Parallel Search to get a Bot's app page. I took before and after screenshots of each flow, in light and dark, after a draft was shown. Nothing wraps badly or overflows.

…ribing one only drafts it

Boundaries, Permissions and the Network policy say their rules are checked exactly as written, the same way every time, and that no model decides. Every describe-in-words field says the model drafts the rule and OpenBot checks what you save, not your words. Approvals only gets the drafting wording, since auto-review and pre-approved rules still ask a reviewer model.
…der the description

On Add rule the CEL field moves up under Describe the rule when a rule is described, and on both Add rule and a rule's page it says: Your model drafted this from your description. This is the rule OpenBot checks before every action, exactly as written. The describe field drops the sentence the label now carries.
…Bot checks

The drafted switches (permission, role or group, on or off) sit above the reading in words and say they are the switches OpenBot checks, exactly as set.
The actions the model chose are listed above the reading, labelled as the only actions the Bot may use once saved, with OpenBot checking every call against the list. An action the app doesn't offer says why under its name.
… rule says the reviewer model decides

The rules sit above the reading and say OpenBot matches each action against them exactly as written. A rule set to Take action if pre-approved says the reviewer model still checks that the person asked for this exact action.
@guidovizoso guidovizoso changed the title Rule screens say saved rules are checked exactly as written Rule screens show what a description became, labelled as what OpenBot checks Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant