From 844503462bc8ecceeba2fe6f965ae3a0ef683f57 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 00:36:11 +0000 Subject: [PATCH 1/2] fix(Handlers): dedupe `included` against the primary data MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `buildIncluded` seeded its `(type, id)` dedup set from `included` alone, so a resource that was both primary data and an include target (the heterogeneous `/search?include=…` case in every example) was emitted twice in one compound document, against JSON:API's one-resource-object- per-pair rule. Seed the set from the primary data too. `Handlers.data` and `Handlers.collection` (and the `Assemble` builders that route through them) now omit `included` when nothing survives that dedup, the same shape as a document handed no `included` at all. Takes `spec/document/no-duplicate-resources` off the known-failure list in all three examples' conformance tests and docs/conformance.md. Closes #147 Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc Co-Authored-By: Claude --- .changeset/dedupe-included-against-primary.md | 5 +++ README.md | 2 +- docs/conformance.md | 5 +-- examples/blog/test/conformance.test.ts | 4 -- examples/github/test/conformance.test.ts | 4 -- examples/northwind/test/conformance.test.ts | 4 -- src/Assemble.test.ts | 16 +++++++ src/Handlers.test.ts | 45 ++++++++++++++++--- src/Handlers.ts | 41 ++++++++++++----- 9 files changed, 93 insertions(+), 33 deletions(-) create mode 100644 .changeset/dedupe-included-against-primary.md diff --git a/.changeset/dedupe-included-against-primary.md b/.changeset/dedupe-included-against-primary.md new file mode 100644 index 00000000..3bf60b2b --- /dev/null +++ b/.changeset/dedupe-included-against-primary.md @@ -0,0 +1,5 @@ +--- +"@thomasfosterau/effect-jsonapi": patch +--- + +`Handlers.data`, `Handlers.collection`, `Handlers.buildIncluded` and the `Assemble` builders that route through them no longer emit a resource in `included` when it is already present as primary data. A compound document now carries one resource object per `(type, id)` pair, as JSON:API requires — previously a heterogeneous collection whose include target was also one of the hits (e.g. `/search?include=author`) repeated that resource in `included`. When every `included` candidate is already primary data, the `included` member is omitted, as it is when none is passed. diff --git a/README.md b/README.md index bbd69520..6b6d3a23 100644 --- a/README.md +++ b/README.md @@ -1145,7 +1145,7 @@ const ArticlesLive = HttpApiBuilder.group(Api, "articles", (handlers) => ) ``` -The document builders (`Handlers.data` / `Handlers.collection`) enforce the compound-document rules at runtime: `included` is **deduplicated** by `(type, id)` and checked for **full linkage** (every included resource must be referenced in the document). Their `onResource` option applies per-resource processing — sparse fieldsets, computed `meta` — uniformly across the whole document ([below](#the-per-resource-transform-pass-onresource)). +The document builders (`Handlers.data` / `Handlers.collection`) enforce the compound-document rules at runtime: `included` is **deduplicated** by `(type, id)` — against itself and against the primary `data`, so a resource that is both a hit and an include target is emitted once, as primary data — and checked for **full linkage** (every included resource must be referenced in the document). Their `onResource` option applies per-resource processing — sparse fieldsets, computed `meta` — uniformly across the whole document ([below](#the-per-resource-transform-pass-onresource)). ### Resolving `included` — one batch per level diff --git a/docs/conformance.md b/docs/conformance.md index 61f0442f..642836bb 100644 --- a/docs/conformance.md +++ b/docs/conformance.md @@ -175,15 +175,14 @@ Two packaging questions are deliberately **not** pre-empted: | Rule | Tracking | Whose | | --- | --- | --- | -| `spec/document/no-duplicate-resources` | [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147) | the library's | | `spec/query/sparse-fieldsets-primary` | [#112](https://github.com/thomasfosterau/effect-jsonapi/issues/112) | the examples' | | `spec/query/sparse-fieldsets-included` | [#112](https://github.com/thomasfosterau/effect-jsonapi/issues/112) | the examples' | | `spec/query/unknown-query-parameter` | [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) | the library's | | `profile/document/jsonapi-member` | [#103](https://github.com/thomasfosterau/effect-jsonapi/issues/103) | the examples' | -The same five in all three examples, which is itself evidence: nothing here is one example's mistake. Two were previously unfiled and were found by the kit — [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147) (`Handlers.buildIncluded` does not dedupe `included` against the primary data) and [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) (unknown query parameters are ignored rather than rejected). A seventh, [#101](https://github.com/thomasfosterau/effect-jsonapi/issues/101), was on this list until its fix landed; the kit now guards it instead. An eighth, [#102](https://github.com/thomasfosterau/effect-jsonapi/issues/102) (the repeated-key spelling was honoured for `include` only), landed the same way. +The same four in all three examples, which is itself evidence: nothing here is one example's mistake. [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) (unknown query parameters are ignored rather than rejected) was previously unfiled and was found by the kit. Three more were on this list until their fixes landed, and the kit now guards them instead: [#101](https://github.com/thomasfosterau/effect-jsonapi/issues/101); [#102](https://github.com/thomasfosterau/effect-jsonapi/issues/102) (the repeated-key spelling was honoured for `include` only); and [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147), also found by the kit (`Handlers.buildIncluded` did not dedupe `included` against the primary data, so a heterogeneous collection's `/search?include=…` repeated a hit in `included`). -Three of the five are now _example_ gaps rather than library ones: #112 landed `onResource` / `Handlers.project` and #103 landed the `jsonapi` builder option, but the worked examples have not adopted either, so the kit still sees unprojected fieldsets and documents that do not advertise their version. Adopting both in `examples/*/handlers.ts` would clear those three — worth doing, and deliberately left to a change that is about the examples rather than about the kit. +Three of the four are now _example_ gaps rather than library ones: #112 landed `onResource` / `Handlers.project` and #103 landed the `jsonapi` builder option, but the worked examples have not adopted either, so the kit still sees unprojected fieldsets and documents that do not advertise their version. Adopting both in `examples/*/handlers.ts` would clear those three — worth doing, and deliberately left to a change that is about the examples rather than about the kit. A new failure means a regression; a disappearing one means a fixed bug whose entry should come off the list. The test says so when either happens. diff --git a/examples/blog/test/conformance.test.ts b/examples/blog/test/conformance.test.ts index 327e27e5..f6e07e03 100644 --- a/examples/blog/test/conformance.test.ts +++ b/examples/blog/test/conformance.test.ts @@ -18,10 +18,6 @@ import { BlogLive } from "../handlers.js"; * Remove an entry when its issue is fixed; the test will tell you to. */ const KNOWN_FAILURES: ReadonlyArray = [ - // #147 — `Handlers.buildIncluded` dedupes `included` against itself but not - // against the primary data, so a resource that is both primary data and an - // include target is emitted twice in one document. A library defect. - ["spec/document/no-duplicate-resources", "#147"], // #112 landed the mechanism — `onResource: Handlers.project(query.fields)` — // but this example's handlers have not adopted it, so `?fields[TYPE]=` still // returns every attribute. An example gap now, not a library one. diff --git a/examples/github/test/conformance.test.ts b/examples/github/test/conformance.test.ts index a7b62dc3..3af4db7a 100644 --- a/examples/github/test/conformance.test.ts +++ b/examples/github/test/conformance.test.ts @@ -18,10 +18,6 @@ import { GitHubLive } from "../handlers.js"; * Remove an entry when its issue is fixed; the test will tell you to. */ const KNOWN_FAILURES: ReadonlyArray = [ - // #147 — `Handlers.buildIncluded` dedupes `included` against itself but not - // against the primary data, so a resource that is both primary data and an - // include target is emitted twice in one document. A library defect. - ["spec/document/no-duplicate-resources", "#147"], // #112 landed the mechanism — `onResource: Handlers.project(query.fields)` — // but this example's handlers have not adopted it, so `?fields[TYPE]=` still // returns every attribute. An example gap now, not a library one. diff --git a/examples/northwind/test/conformance.test.ts b/examples/northwind/test/conformance.test.ts index 2db530a6..b1c00102 100644 --- a/examples/northwind/test/conformance.test.ts +++ b/examples/northwind/test/conformance.test.ts @@ -18,10 +18,6 @@ import { NorthwindLive } from "../handlers.js"; * Remove an entry when its issue is fixed; the test will tell you to. */ const KNOWN_FAILURES: ReadonlyArray = [ - // #147 — `Handlers.buildIncluded` dedupes `included` against itself but not - // against the primary data, so a resource that is both primary data and an - // include target is emitted twice in one document. A library defect. - ["spec/document/no-duplicate-resources", "#147"], // #112 landed the mechanism — `onResource: Handlers.project(query.fields)` — // but this example's handlers have not adopted it, so `?fields[TYPE]=` still // returns every attribute. An example gap now, not a library one. diff --git a/src/Assemble.test.ts b/src/Assemble.test.ts index 2425aaf3..765ea1db 100644 --- a/src/Assemble.test.ts +++ b/src/Assemble.test.ts @@ -392,6 +392,22 @@ describe("Assemble.collection", () => { ).toBeUndefined(); }); + it("does not repeat a page member in `included` (#147)", async () => { + // A heterogeneous page: `place-1` is a hit in its own right and the + // birthplace of the person beside it, reached both by the walk and by the + // caller's own pre-resolved members. + const { targets } = registry([place]); + const document = await run( + Assemble.collection( + [person("a"), place], + { totalCount: 2, offset: 0, limit: 10, baseUrl }, + { included: [place], include: ["birthPlace"], includable: ["birthPlace"], targets }, + ), + ); + expect(ids(document.data)).toEqual(["people:a", "places:place-1"]); + expect(document.included).toBeUndefined(); + }); + it("resolves the page's include walk in one batch per target type, deduplicated", async () => { const { calls, targets } = registry([place]); const document = await run( diff --git a/src/Handlers.test.ts b/src/Handlers.test.ts index dbc06008..0a0e328e 100644 --- a/src/Handlers.test.ts +++ b/src/Handlers.test.ts @@ -75,6 +75,23 @@ describe("Handlers.data", () => { const doc = Handlers.data(article, { included: [stranger], checkLinkage: false }); expect(doc.included).toEqual([stranger]); }); + + it("does not repeat the primary resource in `included` (#147)", () => { + // An article that links to itself: a `related` include resolves back to + // the primary resource, which a document carries once, as `data`. + const selfLinked = { + ...article, + relationships: { + ...article.relationships, + related: { data: [{ type: "articles", id: "1" }] }, + }, + }; + expect(Handlers.data(selfLinked, { included: [selfLinked, author] }).included).toEqual([ + author, + ]); + // …and when it was the only candidate, there is no `included` member at all. + expect(Handlers.data(selfLinked, { included: [selfLinked] })).toEqual({ data: selfLinked }); + }); }); describe("Handlers.collection", () => { @@ -96,6 +113,21 @@ describe("Handlers.collection", () => { expect(Handlers.collection([])).toEqual({ data: [] }); }); + it("does not repeat primary data in `included` for a heterogeneous collection (#147)", () => { + // `/search?include=author`: `people:9` is both a search hit (primary data) + // and the author of another hit. A compound document MUST NOT include more + // than one resource object for each `(type, id)` pair. + const doc = Handlers.collection([article, author], { included: [author, comment] }); + expect(doc.data).toEqual([article, author]); + expect(doc.included).toEqual([comment]); + }); + + it("omits `included` when every candidate is already primary data (#147)", () => { + expect(Handlers.collection([article, author], { included: [author, author] })).toEqual({ + data: [article, author], + }); + }); + it("appends the request's canonical query to the self link", () => { const pairs: ReadonlyArray = [ ["filter[status]", "open"], @@ -390,10 +422,9 @@ describe("the per-resource transform pass", () => { expect(seen).toEqual(["articles:1", "people:9", "comments:5"]); }); - it("runs at most once per (type, id), even for a resource landing in two slots", () => { - // Not a document the builders police, but the transform still cannot - // produce two shapes for one resource: the second slot reuses the first - // slot's result. + it("runs once for a resource offered as both primary data and an `included` member", () => { + // The builders emit it once, as primary data (#147), and shape it once: + // the transform's result for the primary slot is the one the document carries. const selfReferential = { type: "articles", id: "1", @@ -409,7 +440,7 @@ describe("the per-resource transform pass", () => { }, }); expect(runs).toBe(1); - expect(document.included?.[0]).toBe(document.data); + expect(document.included).toBeUndefined(); }); it("runs before the full-linkage check — a transform that breaks linkage is caught", () => { @@ -429,6 +460,10 @@ describe("the per-resource transform pass", () => { ).toEqual([{ type: "articles", id: "2", attributes: relatedArticle.attributes }]); }); + it("buildIncluded drops members already present as primary data (#147)", () => { + expect(Handlers.buildIncluded([article, author], [author, comment, author])).toEqual([comment]); + }); + it("buildIncluded applies it to the deduplicated members", () => { expect( Handlers.buildIncluded([article], [author, author], { diff --git a/src/Handlers.ts b/src/Handlers.ts index a472bcf7..8c0ff3a2 100644 --- a/src/Handlers.ts +++ b/src/Handlers.ts @@ -5,7 +5,9 @@ * helpers assemble those values while enforcing the spec's compound-document * rules at runtime: * - * - `included` resources are deduplicated by `(type, id)` + * - `included` resources are deduplicated by `(type, id)`, against each + * other and against the primary data — a compound document carries one + * resource object per `(type, id)` pair * - full linkage: every included resource must be referenced by a resource * identifier somewhere in the document (data's relationships or another * included resource's relationships) @@ -35,7 +37,7 @@ * * @since 0.1.0 */ -import { Effect } from "effect"; +import { Array as Arr, Effect } from "effect"; import type { JsonApiObject } from "./Document.js"; import { serialise, withPagePairs } from "./internal/canonical.js"; import type { Pair } from "./Query.js"; @@ -568,8 +570,8 @@ const referencedIdentifiers = (resources: ReadonlyArray): Set( onResource: ResourceTransform, ): ResourceTransform => { @@ -598,6 +600,12 @@ const mapResources = ( * transform, and verifies the spec's full-linkage rule: every included * resource must be referenced by a resource identifier in the document. * + * Dedup is document-wide: a resource already present in `primary` is dropped + * from `included` rather than repeated, since a compound document MUST NOT + * include more than one resource object for each `(type, id)` pair. That is + * the case a heterogeneous collection hits whenever an include target is also + * one of the hits. + * * The three steps run in that order deliberately. Dedup first, so `onResource` * runs once per distinct resource rather than once per occurrence; the * transform before the check, so the check sees the resources the document @@ -630,8 +638,9 @@ export const buildIncluded = < readonly onResource?: ResourceTransform | undefined; }, ): ReadonlyArray> => { - // Dedupe by (type, id) - const seen = new Set(); + // Dedupe by (type, id), against the primary data as well as within + // `included`: the one-resource-object-per-pair rule is document-wide. + const seen = new Set(primary.map(key)); const deduped: Array = []; for (const resource of included) { const k = key(resource); @@ -1149,10 +1158,16 @@ const build = ( onResource: ResourceTransform | undefined, ) => { const links = options === undefined ? undefined : buildLinks(options); - const included = - options?.included !== undefined && options.included.length > 0 - ? buildIncluded(primary, options.included, { checkLinkage: options.checkLinkage, onResource }) - : undefined; + // Omitted, not `[]`, when nothing survives dedup against the primary data — + // the same shape as a document that was handed no `included` at all. + const members = + options?.included === undefined + ? [] + : buildIncluded(primary, options.included, { + checkLinkage: options.checkLinkage, + onResource, + }); + const included = Arr.isReadonlyArrayNonEmpty(members) ? members : undefined; return { data, ...(included !== undefined ? { included } : {}), @@ -1165,7 +1180,8 @@ const build = ( /** * Builds a single-resource document value: `{ data, included?, links?, meta?, jsonapi? }`. * - * Included resources are deduplicated and checked for full linkage. + * Included resources are deduplicated — against each other and against the + * primary data — and checked for full linkage. * * Pass `jsonapi` to stamp the top-level `jsonapi` object — `Document.v1_1` to * advertise JSON:API 1.1, or `Atomic.jsonapi` for the atomic-operations @@ -1309,7 +1325,8 @@ export const data = < /** * Builds a collection document value: `{ data: [...], included?, links?, meta?, jsonapi? }`. * - * Included resources are deduplicated and checked for full linkage. + * Included resources are deduplicated — against each other and against the + * primary data — and checked for full linkage. * * JSON:API 1.1 requires a collection's `self` link to carry the query * parameters the client provided. Pass `query` — the request's canonical From 23c920126e9511e660cd12e64c314617e2a72f50 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 00:48:51 +0000 Subject: [PATCH 2/2] test(Assemble): say what the #147 page-member case exercises The include walk already skips resources the document holds, so it never loads `place-1`; only the caller's pre-resolved `included` repeats it. Reword the comment accordingly and assert the walk made no loader call. The changeset now also names the second behaviour change: passing an unreferenced primary resource in `included` (which used to fail the full-linkage check) now succeeds with `included` omitted. Refs #147 Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc Co-Authored-By: Claude --- .changeset/dedupe-included-against-primary.md | 2 +- src/Assemble.test.ts | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/.changeset/dedupe-included-against-primary.md b/.changeset/dedupe-included-against-primary.md index 3bf60b2b..68a24376 100644 --- a/.changeset/dedupe-included-against-primary.md +++ b/.changeset/dedupe-included-against-primary.md @@ -2,4 +2,4 @@ "@thomasfosterau/effect-jsonapi": patch --- -`Handlers.data`, `Handlers.collection`, `Handlers.buildIncluded` and the `Assemble` builders that route through them no longer emit a resource in `included` when it is already present as primary data. A compound document now carries one resource object per `(type, id)` pair, as JSON:API requires — previously a heterogeneous collection whose include target was also one of the hits (e.g. `/search?include=author`) repeated that resource in `included`. When every `included` candidate is already primary data, the `included` member is omitted, as it is when none is passed. +`Handlers.data`, `Handlers.collection`, `Handlers.buildIncluded` and the `Assemble` builders that route through them no longer emit a resource in `included` when it is already present as primary data. A compound document now carries one resource object per `(type, id)` pair, as JSON:API requires — previously a heterogeneous collection whose include target was also one of the hits (e.g. `/search?include=author`) repeated that resource in `included`. When every `included` candidate is already primary data, the `included` member is omitted, as it is when none is passed — so a call that passes a primary resource in `included` with nothing referencing it (e.g. `Handlers.data(article, { included: [article] })`), which used to fail the full-linkage check, now succeeds with `included` omitted. diff --git a/src/Assemble.test.ts b/src/Assemble.test.ts index 765ea1db..aea369e8 100644 --- a/src/Assemble.test.ts +++ b/src/Assemble.test.ts @@ -394,9 +394,11 @@ describe("Assemble.collection", () => { it("does not repeat a page member in `included` (#147)", async () => { // A heterogeneous page: `place-1` is a hit in its own right and the - // birthplace of the person beside it, reached both by the walk and by the - // caller's own pre-resolved members. - const { targets } = registry([place]); + // birthplace of the person beside it. The include walk already skips + // resources the document holds, so it never loads `place-1`; the repeat + // comes only from the caller's own pre-resolved `included`, which is what + // the dedupe against primary data has to drop. + const { calls, targets } = registry([place]); const document = await run( Assemble.collection( [person("a"), place], @@ -404,6 +406,7 @@ describe("Assemble.collection", () => { { included: [place], include: ["birthPlace"], includable: ["birthPlace"], targets }, ), ); + expect(calls).toEqual([]); expect(ids(document.data)).toEqual(["people:a", "places:place-1"]); expect(document.included).toBeUndefined(); });