Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
echo "draft conformance server did not become ready" >&2
exit 1

- name: Run 2026-07-28 server suite
- name: Run 2026-07-28 versioned-spec server suite
run: |
npx -y "@modelcontextprotocol/conformance@${DRAFT_CONFORMANCE_VERSION}" server \
--url http://127.0.0.1:8002/mcp \
Expand All @@ -94,6 +94,32 @@ jobs:
--expected-failures conformance/expected-failures-2026-07-28.yaml \
-o conformance-results

# Extension scenarios are excluded by the --spec-version filter and
# are informational for tiering. Run them explicitly so their gaps remain
# visible and newly passing scenarios make the strict baseline fail stale.
- name: Run Tasks extension server suite (informational)
run: |
scenarios=(
tasks-lifecycle
tasks-capability-negotiation
tasks-wire-fields
tasks-request-state-removal
tasks-mrtr-input
tasks-request-headers
tasks-dispatch-and-envelope
tasks-status-notifications
tasks-required-task-error
tasks-mrtr-composition
)

for scenario in "${scenarios[@]}"; do
npx -y "@modelcontextprotocol/conformance@${DRAFT_CONFORMANCE_VERSION}" server \
--url http://127.0.0.1:8002/mcp \
--scenario "$scenario" \
--expected-failures conformance/expected-failures-extensions.yaml \
-o conformance-extension-results
done

- name: Stop conformance servers
if: always()
run: |
Expand All @@ -105,7 +131,9 @@ jobs:
uses: actions/upload-artifact@v7
with:
name: conformance-server-results
path: conformance-results
path: |
conformance-results
conformance-extension-results

client:
runs-on: ubuntu-latest
Expand All @@ -131,7 +159,7 @@ jobs:
--spec-version 2025-11-25 \
-o conformance-client-results/full

- name: Run 2026-07-28 client suite
- name: Run 2026-07-28 versioned-spec client suite
run: |
npx -y "@modelcontextprotocol/conformance@${DRAFT_CONFORMANCE_VERSION}" client \
--command "$(pwd)/target/debug/conformance-client" \
Expand All @@ -140,6 +168,14 @@ jobs:
--expected-failures conformance/expected-failures-2026-07-28.yaml \
-o conformance-client-results/draft

- name: Run extension client suite (informational)
run: |
npx -y "@modelcontextprotocol/conformance@${DRAFT_CONFORMANCE_VERSION}" client \
--command "$(pwd)/target/debug/conformance-client" \
--suite extensions \
--expected-failures conformance/expected-failures-extensions.yaml \
-o conformance-client-results/extensions

- name: Upload results
if: always()
uses: actions/upload-artifact@v7
Expand Down
26 changes: 19 additions & 7 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
This roadmap tracks the path to SEP-1730 Tier 1 for the Rust MCP SDK.

Spec 2025-11-25 (suite 0.1.16): Server 100% (30/30) · Client 100% (18/18)
Spec 2026-07-28 (suite 0.2.0-alpha.9): Server 92.5% (37/40) · Client 75.0% (24/32)
Spec 2026-07-28 (suite 0.2.0-alpha.9): Server 97.5% (39/40) · Client 90.6% (29/32)

Extension scenarios are reported separately below because they are
informational and do not count toward SDK tiering.

---

Expand All @@ -14,11 +17,20 @@ All 2026-07-28 work carries the `2026-07-28` label and the
Per-scenario conformance status is tracked in the epic issue:
[#977 — Tracking: 2026-07-28 spec conformance](https://github.com/modelcontextprotocol/rust-sdk/issues/977).

### Conformance (baseline 2026-07-13, suite `0.2.0-alpha.9`)
### Versioned-spec conformance (baseline 2026-07-21, suite `0.2.0-alpha.9`)

- Server: 1 expected failure: `json-schema-2020-12`
- Client: 3 expected failures: `tools_call`, `auth/scope-step-up`, and `auth/authorization-server-migration`
- CI: runs the complete `2026-07-28` versioned-spec suites with a strict baseline; an unlisted failure or a listed scenario that starts passing fails the build

### Extension conformance (informational)

Extension-tagged scenarios are excluded by `--spec-version` filters, so CI
runs them in separate server and client steps with
`conformance/expected-failures-extensions.yaml`.

- Server: 3 scenarios (`tools-call-with-progress` stateless behavior, SEP-2243 server-side custom headers, and `server-stateless` — the SEP-2575 discovery/negotiation suite at 2/28 checks)
- Client: 8 scenarios (SEP-2243 headers ×3, `request-metadata`, and 4 single-check auth failures: SEP-2350 step-up, pre-registration, SEP-2352 AS migration, SEP-2468 issuer validation); fixes for SEP-2350 (#888) and SEP-2352 (#965) are already in review
- CI: run the full `--spec-version 2026-07-28` suites (stateless server) instead of hand-picked scenario lists; re-baseline on each draft-suite bump
- SEP-2663 Tasks server: 9 expected failures; `tasks-status-notifications` is currently skipped by the upstream harness; tracked in #868
- Client extensions: `auth/client-credentials-basic` passes; `auth/client-credentials-jwt` and `auth/enterprise-managed-authorization` are expected failures

### Spec features without conformance scenarios

Expand Down Expand Up @@ -100,5 +112,5 @@ These extension scenarios are tracked but do not count toward tier advancement:
|---|---|---|
| `auth/client-credentials-jwt` | extension | ❌ Failed — JWT `aud` claim verification error |
| `auth/client-credentials-basic` | extension | ✅ Passed |
| `auth/cross-app-access-complete-flow` | extension | ❌ Failed — sends `authorization_code` grant instead of `jwt-bearer` |
| `tasks-*` | extension | Not yet attempted |
| `auth/enterprise-managed-authorization` | extension | ❌ Failed — scenario is not implemented by the conformance client |
| `tasks-*` | extension | ❌ 9 expected failures · ⏭️ 1 upstream-skipped scenario |
6 changes: 5 additions & 1 deletion conformance/expected-failures-2026-07-28.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Known failures for the pinned 2026-07-28 draft conformance suite
# (@modelcontextprotocol/conformance DRAFT_CONFORMANCE_VERSION).
#
# The full suites run in CI with `--expected-failures` pointing at this file:
# The full versioned-spec suites run in CI with `--expected-failures` pointing
# at this file. Extension scenarios are filtered out by `--spec-version` and
# tracked separately in `expected-failures-extensions.yaml`.
#
# Within the versioned-spec suites:
# - a scenario failing that is NOT listed here fails the build
# - a scenario listed here that starts passing also fails the build (stale entry),
# so remove it from this list when the underlying issue is fixed.
Expand Down
32 changes: 32 additions & 0 deletions conformance/expected-failures-extensions.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Known failures for informational extension scenarios in
# @modelcontextprotocol/conformance DRAFT_CONFORMANCE_VERSION.
#
# Extensions are not selected by a `--spec-version` run and do not count toward
# SDK tiering. CI runs them separately so a green versioned-spec suite does not
# hide extension gaps.
#
# This is a strict baseline:
# - an unlisted failure fails the build
# - a listed scenario that starts passing fails the build as a stale entry
#
# When bumping DRAFT_CONFORMANCE_VERSION, review the available extension and
# pending scenarios and update this file deliberately.

server:
# SEP-2663 Tasks Extension, tracked in #868.
# `tasks-status-notifications` is intentionally absent: the upstream check is
# currently skipped, and CI should fail if it becomes active but does not pass.
- tasks-lifecycle
- tasks-capability-negotiation
- tasks-wire-fields
- tasks-request-state-removal
- tasks-mrtr-input
- tasks-request-headers
- tasks-dispatch-and-envelope
- tasks-required-task-error
- tasks-mrtr-composition

client:
# Informational OAuth extension scenarios.
- auth/client-credentials-jwt
- auth/enterprise-managed-authorization