Skip to content

test(codemode): run test262 files verbatim from a local checkout - #48446

Merged
rekram1-node merged 1 commit into
v2from
test262-runner
Sep 11, 2026
Merged

test(codemode): run test262 files verbatim from a local checkout#48446
rekram1-node merged 1 commit into
v2from
test262-runner

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Adds a runner that executes upstream test262 files verbatim against the interpreter, so language and built-in behavior is checked against the spec's own tests instead of hand-written paraphrases. The test files themselves are not committed: script/sync-test262.ts copies them from a local checkout at a pinned revision into gitignored directories. Without them the runner registers zero tests, so CI is unchanged.

What's here

  • test/test262/manifest.json — pinned revision (250f204f), directories to copy (built-ins/Array/prototype, language/statements), and what is left out: unsupported flags/features/includes, plus boundaries — code patterns for intentional interpreter limits (classes, this, arguments, call/apply/bind, prototype objects, descriptors, accessors, boxed primitives, sloppy mode, eval, Symbol(), new.target, $262).
  • script/sync-test262.ts — copies from a checkout, verifying it is at the pinned revision and applying the manifest. 3993 files, 5.6 MB, all gitignored.
  • test/test262/run.ts — runs one file: prepends "use strict" (sloppy-only tests are never copied), provides the harness (assert, Test262Error, compareArray, $DONE, $DONOTEVALUATE) as host globals, drains the job queue for async tests, and interprets negative.
  • test/test262.test.ts — one bun test per file present. Files in skipped.txt fail on a known gap and are skipped.
  • script/test262-report.ts — runs everything including skipped files, groups failures by cause, reports skipped files that pass again, and --write regenerates skipped.txt.
  • interpreter-support.md — one [ ] item per gap category found (18).
  • src/interpreter/{runtime,execute}.ts — optional extraGlobals hook so the harness can be host-provided (assert.sameValue is a property on a function, which programs cannot express).

Results at the pinned revision

3252 pass, 741 skipped, 0 fail; ~3 s for the full set.

Largest gaps (all in skipped.txt with reasons): function name/length (238), const { constructor } = err reading undefined (154), async generator parameter evaluation timing (95), interpreter-raised errors typed Error instead of TypeError/RangeError/ReferenceError (94), argument coercion in array methods (42), arr.length = n (21).

Follow-ups: extend the manifest to the remaining directories we implement (Object, JSON, String, Number, Math, Date, Map, Set, Promise, RegExp, language/expressions); decide later whether and how to vendor the files for CI.

@rekram1-node rekram1-node changed the title test(codemode): run vendored test262 files verbatim test(codemode): run test262 files verbatim from a local checkout Sep 11, 2026
@rekram1-node
rekram1-node merged commit 4c34ee5 into v2 Sep 11, 2026
10 checks passed
@rekram1-node
rekram1-node deleted the test262-runner branch September 11, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant