Skip to content

Render the zarr-extensions registry under /extensions/ - #195

Open
normanrz wants to merge 2 commits into
mainfrom
t3/publish-codecs-docs
Open

normanrz wants to merge 2 commits into
mainfrom
t3/publish-codecs-docs

Conversation

@normanrz

@normanrz normanrz commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Publishes the zarr-extensions registry on zarr.dev, e.g. https://zarr.dev/extensions/codecs/jpeg/. zarr-extensions stays the normative source. This site clones it at build time and renders it.

Screenshot 2026-11-04 at 15 27 50

How it works

  • scripts/fetch-extensions.mjs runs before docs:dev and docs:build. It shallow-clones the registry and:

    • writes every README.md as extensions/<path>/index.md, so codecs/jpeg/README.md becomes /extensions/codecs/jpeg/
    • wraps each page in <div v-pre>, because the registry is GitHub markdown, not a Vue template (chunk-key-encodings/template contains {{)
    • rewrites relative links: README.md links and links to directories that have a README become page links; directories without a README (e.g. codecs/zfp/examples/) and missing targets point to GitHub at the imported commit
    • copies every other file into public/extensions/, so schema.json, examples.tsv and the sample_data/*.zarr arrays are served verbatim (e.g. /extensions/codecs/jpeg/schema.json)
    • sets the page title and an editUrl (pointing at the registry repo) in frontmatter, and adds a "Rendered from … at <sha>" note to each page

    Output goes to extensions/ and public/extensions/, which are both gitignored. Use ZARR_EXTENSIONS_DIR to preview a local checkout, or ZARR_EXTENSIONS_REF to build from another branch or tag.

  • .vitepress/extensions.ts builds the /extensions/ sidebar from the generated files: one collapsible group per extension point, with items labelled by registered name.

  • Nav: "Extensions" is added under Resources.

  • editLink.pattern uses frontmatter.editUrl when it is set.

  • deploy.yml now also runs daily and on repository_dispatch with type zarr-extensions-updated.

RTD previews pick this up automatically, since they run npm run docs:build.

Follow-up in zarr-extensions (not part of this PR)

For immediate rebuilds on merge, zarr-extensions needs a small workflow that sends the dispatch event. It needs a token with access to this repo, such as a fine-grained PAT or GitHub App token stored as a secret:

on:
  push:
    branches: [main]
jobs:
  notify:
    runs-on: ubuntu-latest
    steps:
      - run: gh api repos/zarr-developers/zarr-developers.github.io/dispatches -f event_type=zarr-extensions-updated
        env:
          GH_TOKEN: ${{ secrets.WEBSITE_DISPATCH_TOKEN }}

Until that exists, the daily scheduled build picks up changes.

Testing

npm run verify passes: 82 pages and 63 files imported, the redirect check passes, and the internal-link check passes across all pages, including the imported ones. I spot-checked the rendered output for:

  • the page title and h1 on /extensions/codecs/jpeg/
  • literal {{ on the template chunk-key-encoding page
  • ./sample_data/ and ./examples.tsv links, and cross-extension links (e.g. data-types/string → codecs/vlen-utf8)
  • links to directories without a README, which now point to GitHub

🤖 Generated with Claude Code

normanrz and others added 2 commits November 4, 2026 12:29
Clone zarr-developers/zarr-extensions at build time and render each
README.md as a page, e.g. /extensions/codecs/jpeg/. Other registry files
(schema.json, sample data, examples) are served verbatim alongside. The
registry stays the normative source; edit links point there.

The deploy workflow now also runs daily and on a
`zarr-extensions-updated` repository_dispatch event.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@normanrz
normanrz requested a review from maxrjones November 4, 2026 14:47
@normanrz normanrz self-assigned this Nov 4, 2026
@normanrz

normanrz commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

I wonder if we should also render the core specs like this. It would be nice for visual consistency. We would need to be careful to keep all the URLs intact.

@maxrjones maxrjones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing this, @normanrz! I'd prefer a shared VitePress config used by zarr.dev, specs.zarr.dev and extensions.zarr.dev, with each site deployed from its own repo. I think that's a better solution because:

  • it works under the existing governance model where zarr.dev is controlled by ZSC review versus specs.zarr.dev and extensions.zarr.dev being controlled by ZFWG review
  • content from another repo cannot break deploys here
  • this site's build never renders unsanitized Markdown
  • errors in zarr-extensions are caught in that repo's own PR process, using test builds on PRs
  • we don't rely on a cron job to publish extension updates. this setup is fragile because GitHub disables scheduled workflows after 60 days without commits to the repo

That said, having a deployed version of zarr-extensions matters more than the mechanism. I could help (via Claude) with the approach above if others agree, but I'd rather this be merged than have progress stall if my preference is contentious.

@normanrz

normanrz commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

@maxrjones Sounds good. Should we put the VitePress config in a new zarr-developers/vitepress-theme repo, then?
Search would be separate for each subsite, but that seems ok to me.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants