diff --git a/.claude/rules/extension-subtrees.md b/.claude/rules/extension-subtrees.md new file mode 100644 index 00000000000..e235c16de2a --- /dev/null +++ b/.claude/rules/extension-subtrees.md @@ -0,0 +1,30 @@ +--- +paths: + - src/resources/extension-subtrees/** +--- + +# Extension Subtrees + +Files under `src/resources/extension-subtrees//` are git subtrees from external repos. **Do NOT edit directly here.** + +Current subtrees (see [`src/command/dev-call/pull-git-subtree/cmd.ts`](../../src/command/dev-call/pull-git-subtree/cmd.ts) for the source of truth): + +| Name | Upstream repo | +|---|---| +| `julia-engine` | `PumasAI/quarto-julia-engine` | +| `orange-book` | `quarto-ext/orange-book` | + +## Workflow + +1. Open a PR with the fix in the upstream repo +2. Wait for upstream merge +3. From quarto-cli, run: + ```bash + quarto dev-call pull-git-subtree + ``` +4. The command adds two commits (squash + merge) — **do not edit those commits** +5. **Do not rebase** a branch that contains the subtree-pull commits — they cannot be rebased cleanly. Merge to `main` without rebasing, or remove the two commits before rebasing and re-run the pull command. + +## Reference + +Full workflow: [`dev-docs/subtree-extensions.md`](../../dev-docs/subtree-extensions.md) diff --git a/dev-docs/subtree-extensions.md b/dev-docs/subtree-extensions.md index 5ed7f0a03af..d757142c252 100644 --- a/dev-docs/subtree-extensions.md +++ b/dev-docs/subtree-extensions.md @@ -1,6 +1,6 @@ # Subtree extensions -Subtree extensions live in `src/resources/subtree-extensions`. +Subtree extensions live in `src/resources/extension-subtrees`. Each is the complete git repository of a Quarto extension, e.g. the directory tree for the Julia engine looks like @@ -14,6 +14,20 @@ src/resources/extension-subtrees/ ... ``` +The list of subtrees and their upstream repos lives in the `SUBTREES` table in [`src/command/dev-call/pull-git-subtree/cmd.ts`](../src/command/dev-call/pull-git-subtree/cmd.ts). + +## Updating an extension + +Fixes to extension files **must be authored upstream**, never edited directly inside `src/resources/extension-subtrees//`. The correct workflow: + +1. Open a PR with the fix in the upstream repo (e.g. `quarto-ext/orange-book`, `PumasAI/quarto-julia-engine`) +2. Merge the PR upstream +3. From quarto-cli, run the pull command (see [Pulling subtree updates](#pulling-subtree-updates)) + +Editing directly inside the subtree prefix is wrong: the change will be overwritten the next time the subtree is pulled, and anyone who forked the upstream extension will not benefit from it. + +## Pulling subtree updates + The command to add or update a subtree is ``` @@ -39,3 +53,37 @@ So you must either - run the command when ready to merge to main, or - remove the commits when rebasing, and run the `dev-call` command again + +## Recovery: edits made in-tree by mistake + +If a fix has already been committed directly inside the subtree prefix (wrong workflow), it needs to be re-authored upstream before resuming the normal flow. Two paths: + +### Option A — re-author upstream manually (recommended) + +Faster and more predictable, especially in repos with many commits where `git subtree split` walks the full history. + +1. Clone the upstream repo locally (e.g. `git clone https://github.com/quarto-ext/orange-book.git`) +2. Create a feature branch from upstream `main` +3. Apply the same change at the upstream repo's path (the subtree's `_extensions/...` becomes the repo root) +4. Commit (preserve original author and message) and push to the upstream repo +5. Open a PR upstream and merge + +### Option B — `git subtree push` + +Extracts the prefix-scoped history into a synthetic branch and pushes it. Works but is slow on large repos (the split walks every commit) and untested in this codebase. + +``` +git fetch +git subtree push --prefix=src/resources/extension-subtrees/ \ + +``` + +Then open a PR upstream from ``. + +### After the upstream PR merges + +Back in quarto-cli, on the branch holding the original in-tree commit: + +1. Remove the in-tree edit from the local commit (amend or rebase out the lines under the subtree prefix) so `pull-git-subtree` does not conflict with itself +2. Run `quarto dev-call pull-git-subtree ` to bring the upstream change in via the normal two-commit pattern +3. Merge the branch to `main` without rebasing (the subtree-pull commits cannot be rebased — see [Pulling subtree updates](#pulling-subtree-updates)) diff --git a/llm-docs/localization-architecture.md b/llm-docs/localization-architecture.md new file mode 100644 index 00000000000..0d6d4651860 --- /dev/null +++ b/llm-docs/localization-architecture.md @@ -0,0 +1,222 @@ +--- +main_commit: a8d0dcfee +analyzed_date: 2026-05-18 +key_files: + - src/resources/language/_language.yml + - src/resources/language/_language-fr.yml + - src/core/language.ts + - src/command/render/render-contexts.ts + - src/command/render/pandoc.ts + - src/command/render/filters.ts + - src/resources/filters/common/meta.lua + - src/resources/filters/crossref/meta.lua + - src/resources/filters/crossref/refs.lua + - src/resources/filters/crossref/format.lua + - src/resources/filters/modules/authors.lua + - src/resources/filters/modules/callouts.lua + - src/resources/filters/layout/meta.lua + - src/resources/formats/html/pandoc/html.template + - src/resources/formats/html/pandoc/title-block.html + - src/resources/formats/html/templates/title-metadata.html + - src/resources/formats/pdf/pandoc/latex.template + - src/resources/formats/pdf/pandoc/babel-lang.tex + - src/resources/formats/pdf/pandoc/toc.tex + - src/resources/formats/beamer/pandoc/beamer.template + - src/resources/formats/typst/pandoc/typst.template + - src/resources/formats/typst/pandoc/quarto/typst-template.typ + - src/resources/formats/typst/pandoc/quarto/typst-show.typ + - src/resources/extension-subtrees/orange-book/_extensions/orange-book/typst-show.typ + - src/format/html/format-html-shared.ts + - src/format/html/format-html-bootstrap.ts + - src/format/html/format-html-appendix.ts + - src/format/pdf/format-pdf.ts + - src/project/types/book/book-chapters.ts + - src/project/types/website/listing/website-listing-template.ts + - src/project/types/website/website-search.ts +--- + +# Localization architecture + +How Quarto turns `lang: fr` (and similar) into localized strings across HTML, LaTeX/PDF, Typst, and the various extension points. Required reading before adding or wiring a new translatable string. + +## TL;DR map + +``` +_language.yml + _language-.yml + │ + ▼ +core/language.ts (formatLanguage, translationsForLang) + │ + ▼ +format.language (per-render Format object) + │ + ├─► languageFilterParams (filters.ts) + │ └─► Lua filter params (JSON) + │ └─► param("key", default) in Lua filters + │ ├─► written into Pandoc AST as text + │ └─► written into meta (becomes $var$ in templates) + │ + ├─► explicit copies into format.metadata (pandoc.ts) + │ └─► Pandoc YAML metadata + │ └─► $var$ in Pandoc templates + │ + └─► direct TS reads (HTML format extras, listings, search, + appendices) writing into rendered DOM / config +``` + +Plus format-native localization (LaTeX babel, Typst `set text(lang:)`) that Quarto does NOT control — the language just flows as a value Pandoc/Typst/babel know to interpret. + +## 1. Source of truth and resolution + +- `src/resources/language/_language.yml` — base English defaults for every Quarto-defined localized key. +- `src/resources/language/_language-.yml` — overrides per language. `_language-fr.yml` provides French; `_language-pt-BR.yml` provides Brazilian Portuguese with `_language-pt.yml` as parent. +- `src/resources/schema/definitions.yml` — `format-language` schema enumerates valid keys. Adding a new key to YAML requires also adding it here. +- `src/config/constants.ts:320-383` — `kCrossref*Title`, `kCrossref*Prefix`, `k*Title`, `kListingPage*` constants for every key, plus `kLanguageDefaultsKeys` array used as the gatekeeper. +- `src/core/language.ts:135` — `readDefaultLanguageTranslations(lang)` resolves the base file, calls `readLanguageTranslations` which walks BCP-47 subtags and merges variation files. +- `src/core/language.ts:103-130` — `readLanguageTranslations`: per subtag, loads `_language-.yml` and merges onto base. +- `src/core/language.ts:161-188` — `translationsForLang(language, lang)`: filters to `kLanguageDefaultsKeys`, `crossref-*-title`, `crossref-*-prefix`, then merges variation sub-objects (`fr-CA` falls back to `fr`). +- `src/core/language.ts:190-211` — `formatLanguage(metadata, language, flags)`: reads `kLang` from flags/metadata (default `"en"`), calls `readDefaultLanguageTranslations`, merges any user-provided `language:` block, calls `translationsForLang`. +- `src/command/render/render-contexts.ts:254-258` — assigns the resolved object onto every `Format`: + ```ts + formats[formatKey].format.language = await formatLanguage( + formats[formatKey].format.metadata, + formats[formatKey].format.language, + options.flags, + ); + ``` + +After this point, `format.language` is the in-memory canonical map of `{ "crossref-ch-prefix": "Chapitre", "toc-title-document": "Table des matières", ... }`. + +## 2. Three downstream surfaces from `format.language` + +`format.language` is not consumed directly by Pandoc. Three distinct pipelines plug into it. + +### 2a. Lua filter params (the broad channel) + +`src/command/render/filters.ts:465-496` (`languageFilterParams`) auto-spreads selected language keys into Lua filter params: + +- Always: `[kCodeSummary]`, `[kTocTitleDocument]`. +- For each crossref type (`fig`, `tbl`, `lst`, `thm`, ...): `params["crossref--prefix"] = language["crossref--title"]` — the prefix-from-title default. +- Bulk: every key in `format.language` that starts with `callout-`, `crossref-`, or `environment-` is copied verbatim. + +`src/command/render/pandoc.ts:492` separately puts the whole language table into filter params at key `"language"`: + +```ts +formatFilterParams["language"] = options.format.language; +``` + +Both routes coexist in filter params JSON. Lua filters access them via: +- `param("language", nil)["title-block-author-single"]` — convenient when reading many keys (used in `modules/authors.lua:854`). +- `param("crossref-ch-prefix", "Chapter")` — convenient with default fallback (used throughout `crossref/`). + +Same underlying data, just different access ergonomics. Pick the style that matches the file you are editing. + +### 2b. Direct `format.metadata` writes in TS + +`format.metadata` becomes Pandoc YAML metadata, so its keys resolve as `$key$` in Pandoc templates. Quarto explicitly copies a few language values across in `src/command/render/pandoc.ts`: + +- `pandoc.ts:498-503` — `format.metadata[kTocTitle]` from website or document variant of toc title. +- `pandoc.ts:520-522` — `format.metadata[kAbstractTitle]` from `kSectionTitleAbstract`. + +Both are gated "only if user did not already set the metadata key". + +### 2c. Direct TS reads (`format.language[key]` in renderer code) + +Some HTML/PDF rendering code reads `format.language[...]` directly to populate DOM nodes or JS configs rather than going through Pandoc metadata: + +- `src/format/html/format-html-shared.ts:356,372,478` — footnotes/references headings, copy-button tooltip. +- `src/format/html/format-html-bootstrap.ts:711,735,753,757,905` — dev container, Binder, Other Links, Code Links, Related Formats. +- `src/format/html/format-html-appendix.ts:240,252,282,301,318,327` — license, reuse, copyright, BibTeX attribution, cite-as, citation labels. +- `src/format/asciidoc/format-asciidoc.ts:244` — references section. +- `src/project/types/website/website-search.ts:613-617` — copies all `search-*` keys into search JS options. +- `src/project/types/website/listing/website-listing-read.ts:153-163` — listing column header labels. +- `src/project/types/website/listing/website-listing-template.ts:343,353,366,375` — listing sort UI. +- `src/project/types/book/book-chapters.ts:149` — uses `kCrossrefApxPrefix` to format appendix chapter titles ("Annexe A — ..."). +- `src/render/notebook/notebook-contributor-html.ts:120,174,208,209` — notebook preview UI labels. +- `src/command/render/codetools.ts:211,215,221,265,330` — code tools menu labels. + +These bypass both Pandoc metadata and Lua filters. The string lands directly in the rendered DOM or in a JSON blob the front-end JS reads. + +## 3. Format-by-format breakdown + +### 3a. HTML + +Localization paths used: + +- **Document `lang` attribute**: `_language.yml` lookup not involved. `lang:` flows as Pandoc-native metadata, template renders `` (`html.template:2`). +- **TOC title**: `$toc-title$` resolved via channel 2b. Used in `html.template:60`, `toc.html:3`. +- **Abstract title**: `$abstract-title$` resolved via 2b. Used in `html.template:51`, `title-block.html:15`. +- **Title block labels** (`Authors`, `Affiliations`, `Published`, `Modified`, `Doi`, `Abstract`, `Keywords`): `$labels.*$` written into meta by `modules/authors.lua:854-913` (`computeLabels`). Templates: `title-metadata.html:3,4,32,43,52,61,74,83`, `manuscript/title-metadata.html:6,7,37,48,57,66,83,92`. +- **Crossref text** (`Figure 1.1`, `Table 2.1`): assembled in Lua by `crossref/format.lua` using `title()` / `refPrefix()` which call `param("crossref--title"/"-prefix")`. Written as inline text directly into the AST. By the time Pandoc renders HTML, the localized prefix is already document content. +- **Callout titles** (`Tip`, `Note`, etc.): `modules/callouts.lua:15,185` reads `param("callout--title", default)`, writes into the callout node. +- **DOM-level UI strings** (search, listings, related formats, code tools, etc.): TS surface (2c) writes into the DOM in HTML postprocessors. + +### 3b. LaTeX / PDF + +Three localization paths layered: + +1. **babel (Pandoc-native)** — `_language-.yml` not involved. Pandoc converts `lang:` into `$babel-lang$` and friends; `pdf/pandoc/babel-lang.tex:4-23` injects `\usepackage{babel}` with the right option; `babel` then automatically localizes `\chaptername`, `\figurename`, `\tablename`, `\contentsname`, `\listfigurename`, `\listtablename`, `\proofname`. These are "free" — no Quarto code touches them. + +2. **Lua-injected `\renewcommand` overrides** — `crossref/meta.lua:32-43` calls `metaInjectLatex` (defined at `common/meta.lua:51-59`, format-gated to LaTeX), which appends raw TeX to `header-includes`. Uses `maybeRenewCommand` (`crossref/meta.lua:93-95`) which emits `\ifdefined\\renewcommand*\{arg}\else\newcommand\{arg}\fi`. This deliberately overrides babel's defaults with the Quarto-side crossref title (so user can override e.g. `figurename` via crossref options). Overridden commands: `contentsname`, `listfigurename`, `listtablename`, `figurename`, `tablename`. `crossref/theorems.lua:217` does the same for `\proofname` via raw `\renewcommand` inside `\AtBeginDocument`. + + Other Lua filters use `metaInjectLatex` for non-language LaTeX customization (`crossref/custom.lua:78`, `layout/meta.lua`, `quarto-post/landscape.lua`, etc. — they inject packages or styling, not localized strings). + +3. **Pandoc template `$var$`** — `pdf/pandoc/toc.tex:3` and `pdf/pandoc/latex.template:93-94` use `$toc-title$` to set `\contentsname`. Beamer templates (`beamer/pandoc/beamer.template:146-151`, `beamer/pandoc/toc.tex:3`) likewise. + +PDF-side TS code does not read `format.language` directly except `src/format/pdf/format-pdf.ts:242` which registers `"babel-lang"` as a Quarto partial. + +### 3c. Typst + +Localization paths: + +1. **Typst native (`set text(lang: ...)`)** — `lang:` flows as Pandoc metadata, `$lang$` resolves to `lang: "fr"` in `typst-show.typ:22-24`, then `typst-template.typ:45-47` applies `set text(lang: lang, region: region, size: fontsize)`. Typst then auto-localizes its built-in figure/table/equation/listing/appendix/bibliography/outline strings using its own translation tables. Like babel for LaTeX, Quarto does not control these — they come for free from Typst. + +2. **Pandoc template `$var$` via meta** — same channel as HTML. + - `$toc-title$` (resolved via 2b in `pandoc.ts:498`) used in `typst-show.typ:93-95` as `toc_title:`. + - `$labels.abstract$` (written by `authors.lua` `computeLabels`) used in `typst-show.typ:30` as `abstract-title:`. + - For orange-book (`extension-subtrees/orange-book/_extensions/orange-book/typst-show.typ:28-33`): `$crossref-lof-title$`, `$crossref-lot-title$`, `$crossref-ch-prefix$` used to set `list-of-figure-title`, `list-of-table-title`, `supplement-chapter` parameters on `book.with(...)`. These are surfaced into meta by `crossref/meta.lua:6-23` (the recent fix — see §4 below). + +3. **Lua-injected supplements for refs** — `crossref/refs.lua:89-91` wraps every `@fig-foo`-style reference into `#ref(