Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/dedupe-included-against-primary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@thomasfosterau/effect-jsonapi": patch
---

`Handlers.data`, `Handlers.collection`, `Handlers.buildIncluded` and the `Assemble` builders that route through them no longer emit a resource in `included` when it is already present as primary data. A compound document now carries one resource object per `(type, id)` pair, as JSON:API requires — previously a heterogeneous collection whose include target was also one of the hits (e.g. `/search?include=author`) repeated that resource in `included`. When every `included` candidate is already primary data, the `included` member is omitted, as it is when none is passed — so a call that passes a primary resource in `included` with nothing referencing it (e.g. `Handlers.data(article, { included: [article] })`), which used to fail the full-linkage check, now succeeds with `included` omitted.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1145,7 +1145,7 @@ const ArticlesLive = HttpApiBuilder.group(Api, "articles", (handlers) =>
)
```

The document builders (`Handlers.data` / `Handlers.collection`) enforce the compound-document rules at runtime: `included` is **deduplicated** by `(type, id)` and checked for **full linkage** (every included resource must be referenced in the document). Their `onResource` option applies per-resource processing — sparse fieldsets, computed `meta` — uniformly across the whole document ([below](#the-per-resource-transform-pass-onresource)).
The document builders (`Handlers.data` / `Handlers.collection`) enforce the compound-document rules at runtime: `included` is **deduplicated** by `(type, id)` — against itself and against the primary `data`, so a resource that is both a hit and an include target is emitted once, as primary data — and checked for **full linkage** (every included resource must be referenced in the document). Their `onResource` option applies per-resource processing — sparse fieldsets, computed `meta` — uniformly across the whole document ([below](#the-per-resource-transform-pass-onresource)).

### Resolving `included` — one batch per level

Expand Down
5 changes: 2 additions & 3 deletions docs/conformance.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,15 +175,14 @@ Two packaging questions are deliberately **not** pre-empted:

| Rule | Tracking | Whose |
| --- | --- | --- |
| `spec/document/no-duplicate-resources` | [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147) | the library's |
| `spec/query/sparse-fieldsets-primary` | [#112](https://github.com/thomasfosterau/effect-jsonapi/issues/112) | the examples' |
| `spec/query/sparse-fieldsets-included` | [#112](https://github.com/thomasfosterau/effect-jsonapi/issues/112) | the examples' |
| `spec/query/unknown-query-parameter` | [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) | the library's |
| `profile/document/jsonapi-member` | [#103](https://github.com/thomasfosterau/effect-jsonapi/issues/103) | the examples' |

The same five in all three examples, which is itself evidence: nothing here is one example's mistake. Two were previously unfiled and were found by the kit — [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147) (`Handlers.buildIncluded` does not dedupe `included` against the primary data) and [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) (unknown query parameters are ignored rather than rejected). A seventh, [#101](https://github.com/thomasfosterau/effect-jsonapi/issues/101), was on this list until its fix landed; the kit now guards it instead. An eighth, [#102](https://github.com/thomasfosterau/effect-jsonapi/issues/102) (the repeated-key spelling was honoured for `include` only), landed the same way.
The same four in all three examples, which is itself evidence: nothing here is one example's mistake. [#148](https://github.com/thomasfosterau/effect-jsonapi/issues/148) (unknown query parameters are ignored rather than rejected) was previously unfiled and was found by the kit. Three more were on this list until their fixes landed, and the kit now guards them instead: [#101](https://github.com/thomasfosterau/effect-jsonapi/issues/101); [#102](https://github.com/thomasfosterau/effect-jsonapi/issues/102) (the repeated-key spelling was honoured for `include` only); and [#147](https://github.com/thomasfosterau/effect-jsonapi/issues/147), also found by the kit (`Handlers.buildIncluded` did not dedupe `included` against the primary data, so a heterogeneous collection's `/search?include=…` repeated a hit in `included`).

Three of the five are now _example_ gaps rather than library ones: #112 landed `onResource` / `Handlers.project` and #103 landed the `jsonapi` builder option, but the worked examples have not adopted either, so the kit still sees unprojected fieldsets and documents that do not advertise their version. Adopting both in `examples/*/handlers.ts` would clear those three — worth doing, and deliberately left to a change that is about the examples rather than about the kit.
Three of the four are now _example_ gaps rather than library ones: #112 landed `onResource` / `Handlers.project` and #103 landed the `jsonapi` builder option, but the worked examples have not adopted either, so the kit still sees unprojected fieldsets and documents that do not advertise their version. Adopting both in `examples/*/handlers.ts` would clear those three — worth doing, and deliberately left to a change that is about the examples rather than about the kit.

A new failure means a regression; a disappearing one means a fixed bug whose entry should come off the list. The test says so when either happens.

Expand Down
4 changes: 0 additions & 4 deletions examples/blog/test/conformance.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@ import { BlogLive } from "../handlers.js";
* Remove an entry when its issue is fixed; the test will tell you to.
*/
const KNOWN_FAILURES: ReadonlyArray<readonly [id: string, tracking: string]> = [
// #147 — `Handlers.buildIncluded` dedupes `included` against itself but not
// against the primary data, so a resource that is both primary data and an
// include target is emitted twice in one document. A library defect.
["spec/document/no-duplicate-resources", "#147"],
// #112 landed the mechanism — `onResource: Handlers.project(query.fields)` —
// but this example's handlers have not adopted it, so `?fields[TYPE]=` still
// returns every attribute. An example gap now, not a library one.
Expand Down
4 changes: 0 additions & 4 deletions examples/github/test/conformance.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@ import { GitHubLive } from "../handlers.js";
* Remove an entry when its issue is fixed; the test will tell you to.
*/
const KNOWN_FAILURES: ReadonlyArray<readonly [id: string, tracking: string]> = [
// #147 — `Handlers.buildIncluded` dedupes `included` against itself but not
// against the primary data, so a resource that is both primary data and an
// include target is emitted twice in one document. A library defect.
["spec/document/no-duplicate-resources", "#147"],
// #112 landed the mechanism — `onResource: Handlers.project(query.fields)` —
// but this example's handlers have not adopted it, so `?fields[TYPE]=` still
// returns every attribute. An example gap now, not a library one.
Expand Down
4 changes: 0 additions & 4 deletions examples/northwind/test/conformance.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@ import { NorthwindLive } from "../handlers.js";
* Remove an entry when its issue is fixed; the test will tell you to.
*/
const KNOWN_FAILURES: ReadonlyArray<readonly [id: string, tracking: string]> = [
// #147 — `Handlers.buildIncluded` dedupes `included` against itself but not
// against the primary data, so a resource that is both primary data and an
// include target is emitted twice in one document. A library defect.
["spec/document/no-duplicate-resources", "#147"],
// #112 landed the mechanism — `onResource: Handlers.project(query.fields)` —
// but this example's handlers have not adopted it, so `?fields[TYPE]=` still
// returns every attribute. An example gap now, not a library one.
Expand Down
19 changes: 19 additions & 0 deletions src/Assemble.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,25 @@ describe("Assemble.collection", () => {
).toBeUndefined();
});

it("does not repeat a page member in `included` (#147)", async () => {
// A heterogeneous page: `place-1` is a hit in its own right and the
// birthplace of the person beside it. The include walk already skips
// resources the document holds, so it never loads `place-1`; the repeat
// comes only from the caller's own pre-resolved `included`, which is what
// the dedupe against primary data has to drop.
const { calls, targets } = registry([place]);
const document = await run(
Assemble.collection(
[person("a"), place],
{ totalCount: 2, offset: 0, limit: 10, baseUrl },
{ included: [place], include: ["birthPlace"], includable: ["birthPlace"], targets },
),
);
expect(calls).toEqual([]);
expect(ids(document.data)).toEqual(["people:a", "places:place-1"]);
expect(document.included).toBeUndefined();
});

it("resolves the page's include walk in one batch per target type, deduplicated", async () => {
const { calls, targets } = registry([place]);
const document = await run(
Expand Down
45 changes: 40 additions & 5 deletions src/Handlers.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,23 @@ describe("Handlers.data", () => {
const doc = Handlers.data(article, { included: [stranger], checkLinkage: false });
expect(doc.included).toEqual([stranger]);
});

it("does not repeat the primary resource in `included` (#147)", () => {
// An article that links to itself: a `related` include resolves back to
// the primary resource, which a document carries once, as `data`.
const selfLinked = {
...article,
relationships: {
...article.relationships,
related: { data: [{ type: "articles", id: "1" }] },
},
};
expect(Handlers.data(selfLinked, { included: [selfLinked, author] }).included).toEqual([
author,
]);
// …and when it was the only candidate, there is no `included` member at all.
expect(Handlers.data(selfLinked, { included: [selfLinked] })).toEqual({ data: selfLinked });
});
});

describe("Handlers.collection", () => {
Expand All @@ -96,6 +113,21 @@ describe("Handlers.collection", () => {
expect(Handlers.collection([])).toEqual({ data: [] });
});

it("does not repeat primary data in `included` for a heterogeneous collection (#147)", () => {
// `/search?include=author`: `people:9` is both a search hit (primary data)
// and the author of another hit. A compound document MUST NOT include more
// than one resource object for each `(type, id)` pair.
const doc = Handlers.collection([article, author], { included: [author, comment] });
expect(doc.data).toEqual([article, author]);
expect(doc.included).toEqual([comment]);
});

it("omits `included` when every candidate is already primary data (#147)", () => {
expect(Handlers.collection([article, author], { included: [author, author] })).toEqual({
data: [article, author],
});
});

it("appends the request's canonical query to the self link", () => {
const pairs: ReadonlyArray<Query.Pair> = [
["filter[status]", "open"],
Expand Down Expand Up @@ -390,10 +422,9 @@ describe("the per-resource transform pass", () => {
expect(seen).toEqual(["articles:1", "people:9", "comments:5"]);
});

it("runs at most once per (type, id), even for a resource landing in two slots", () => {
// Not a document the builders police, but the transform still cannot
// produce two shapes for one resource: the second slot reuses the first
// slot's result.
it("runs once for a resource offered as both primary data and an `included` member", () => {
// The builders emit it once, as primary data (#147), and shape it once:
// the transform's result for the primary slot is the one the document carries.
const selfReferential = {
type: "articles",
id: "1",
Expand All @@ -409,7 +440,7 @@ describe("the per-resource transform pass", () => {
},
});
expect(runs).toBe(1);
expect(document.included?.[0]).toBe(document.data);
expect(document.included).toBeUndefined();
});

it("runs before the full-linkage check — a transform that breaks linkage is caught", () => {
Expand All @@ -429,6 +460,10 @@ describe("the per-resource transform pass", () => {
).toEqual([{ type: "articles", id: "2", attributes: relatedArticle.attributes }]);
});

it("buildIncluded drops members already present as primary data (#147)", () => {
expect(Handlers.buildIncluded([article, author], [author, comment, author])).toEqual([comment]);
});

it("buildIncluded applies it to the deduplicated members", () => {
expect(
Handlers.buildIncluded([article], [author, author], {
Expand Down
41 changes: 29 additions & 12 deletions src/Handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
* helpers assemble those values while enforcing the spec's compound-document
* rules at runtime:
*
* - `included` resources are deduplicated by `(type, id)`
* - `included` resources are deduplicated by `(type, id)`, against each
* other and against the primary data — a compound document carries one
* resource object per `(type, id)` pair
* - full linkage: every included resource must be referenced by a resource
* identifier somewhere in the document (data's relationships or another
* included resource's relationships)
Expand Down Expand Up @@ -35,7 +37,7 @@
*
* @since 0.1.0
*/
import { Effect } from "effect";
import { Array as Arr, Effect } from "effect";
import type { JsonApiObject } from "./Document.js";
import { serialise, withPagePairs } from "./internal/canonical.js";
import type { Pair } from "./Query.js";
Expand Down Expand Up @@ -568,8 +570,8 @@ const referencedIdentifiers = (resources: ReadonlyArray<ResourceValue>): Set<str

// Wraps a transform so it runs at most once per `(type, id)`, handing back
// the same value everywhere that resource appears in the document. A resource
// that is both primary data and an `included` member is therefore shaped
// once, not twice — it cannot come out two ways.
// offered as both primary data and an `included` member is shaped once, for
// the primary slot, and `buildIncluded` then drops it from `included`.
const memoise = <F extends TransformLambda>(
onResource: ResourceTransform<F>,
): ResourceTransform<F> => {
Expand Down Expand Up @@ -598,6 +600,12 @@ const mapResources = <F extends TransformLambda>(
* transform, and verifies the spec's full-linkage rule: every included
* resource must be referenced by a resource identifier in the document.
*
* Dedup is document-wide: a resource already present in `primary` is dropped
* from `included` rather than repeated, since a compound document MUST NOT
* include more than one resource object for each `(type, id)` pair. That is
* the case a heterogeneous collection hits whenever an include target is also
* one of the hits.
*
* The three steps run in that order deliberately. Dedup first, so `onResource`
* runs once per distinct resource rather than once per occurrence; the
* transform before the check, so the check sees the resources the document
Expand Down Expand Up @@ -630,8 +638,9 @@ export const buildIncluded = <
readonly onResource?: ResourceTransform<F> | undefined;
},
): ReadonlyArray<Transformed<F, Included>> => {
// Dedupe by (type, id)
const seen = new Set<string>();
// Dedupe by (type, id), against the primary data as well as within
// `included`: the one-resource-object-per-pair rule is document-wide.
const seen = new Set<string>(primary.map(key));
const deduped: Array<Included> = [];
for (const resource of included) {
const k = key(resource);
Expand Down Expand Up @@ -1149,10 +1158,16 @@ const build = (
onResource: ResourceTransform<TransformLambda> | undefined,
) => {
const links = options === undefined ? undefined : buildLinks(options);
const included =
options?.included !== undefined && options.included.length > 0
? buildIncluded(primary, options.included, { checkLinkage: options.checkLinkage, onResource })
: undefined;
// Omitted, not `[]`, when nothing survives dedup against the primary data —
// the same shape as a document that was handed no `included` at all.
const members =
options?.included === undefined
? []
: buildIncluded(primary, options.included, {
checkLinkage: options.checkLinkage,
onResource,
});
const included = Arr.isReadonlyArrayNonEmpty(members) ? members : undefined;
return {
data,
...(included !== undefined ? { included } : {}),
Expand All @@ -1165,7 +1180,8 @@ const build = (
/**
* Builds a single-resource document value: `{ data, included?, links?, meta?, jsonapi? }`.
*
* Included resources are deduplicated and checked for full linkage.
* Included resources are deduplicated — against each other and against the
* primary data — and checked for full linkage.
*
* Pass `jsonapi` to stamp the top-level `jsonapi` object — `Document.v1_1` to
* advertise JSON:API 1.1, or `Atomic.jsonapi` for the atomic-operations
Expand Down Expand Up @@ -1309,7 +1325,8 @@ export const data = <
/**
* Builds a collection document value: `{ data: [...], included?, links?, meta?, jsonapi? }`.
*
* Included resources are deduplicated and checked for full linkage.
* Included resources are deduplicated — against each other and against the
* primary data — and checked for full linkage.
*
* JSON:API 1.1 requires a collection's `self` link to carry the query
* parameters the client provided. Pass `query` — the request's canonical
Expand Down
Loading