Skip to content

feat: include headers from portable deps for native gem compilation#1

Merged
jdx merged 6 commits intomainfrom
feat/include-headers-for-native-gems
Dec 19, 2025
Merged

feat: include headers from portable deps for native gem compilation#1
jdx merged 6 commits intomainfrom
feat/include-headers-for-native-gems

Conversation

@jdx
Copy link
Copy Markdown
Owner

@jdx jdx commented Dec 19, 2025

Summary

When using precompiled Ruby binaries, gems with native extensions (like openssl, psych, ffi) fail to compile because headers are missing from the tarball.

This change:

  1. Copies headers from portable dependencies (openssl, libyaml, libffi, zlib, libxcrypt) into the Ruby tarball's include/ directory during packaging
  2. Updates rbconfig.rb to include -I#{prefix}/include in CPPFLAGS and -L#{prefix}/lib in LDFLAGS so mkmf can find the headers when building native extensions

Background

Reported in mise discussion: jdx/mise#7268 (comment)

Users on macOS with precompiled Ruby were unable to update gems because native extensions couldn't find necessary headers:

  • cannot find openssl/ssl.h
  • cannot find yaml.h

Changes

  • cmd/jdx-package.rb: Added copy_portable_headers method that copies headers from installed portable dependencies into the extracted Ruby directory before re-tarballing
  • Abstract/jdx-ruby.rb: Added inreplace block to patch rbconfig.rb with include/lib paths

Test plan

  • CI builds pass on macOS and Linux
  • Verify headers are present in extracted tarball: tar -tzf ruby-3.4.1.macos.tar.gz | grep include
  • Test native gem installation: gem install openssl psych

🤖 Generated with Claude Code


Note

Bundles headers/static libs/pkg-config from portable deps into Ruby formulas and updates CI to build on PRs with a required status check.

  • Portable Ruby formulas (Abstract/jdx-ruby*.rb):
    • Copy portable dependency assets for native gem builds via copy_portable_deps_for_native_gems (headers, static libs, pkg-config); set portable_deps (adds libffi, zlib, libxcrypt on Linux).
    • Remove shipping libxcrypt's libcrypt.a directly.
    • Tests: set ENV["PKG_CONFIG_PATH"] = "lib/pkgconfig"; install native gems (openssl, psych) to validate headers; keep existing byebug test.
    • Update bundled bootsnap gem version in jdx-ruby-34.rb and Abstract/jdx-ruby.rb.
  • Portable mixin (Abstract/portable-formula.rb):
    • Add copy_portable_deps_for_native_gems helper to copy headers, *.a, and patch .pc files with relocatable prefix.
  • CI (.github/workflows/tests.yml):
    • Add build job for PRs using ./.github/workflows/build.yml with formula: jdx-ruby@3.4.1.
    • Add conclusion job to surface a single required status check.

Written by Cursor Bugbot for commit 82c1061. This will update automatically on new commits. Configure here.

@jdx jdx force-pushed the feat/include-headers-for-native-gems branch 2 times, most recently from 2e14522 to f1d90ab Compare December 19, 2025 15:34
@jdx jdx force-pushed the feat/include-headers-for-native-gems branch from f1d90ab to 1fe835a Compare December 19, 2025 16:24
When using precompiled Ruby binaries, gems with native extensions
(like openssl, psych, ffi) fail to compile because headers are missing.

This change copies headers from portable dependencies (openssl, libyaml,
etc.) into the Ruby installation's include/ directory during the Homebrew
install phase. This ensures:

1. Headers are available during `brew test` (after deps are uninstalled)
2. Headers are included in the final tarball
3. Paths are relocatable (uses Ruby's standard includedir, not hardcoded)

Also adds CI tests for openssl and psych gem installation.

Fixes: jdx/mise#7268 (comment)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@jdx jdx force-pushed the feat/include-headers-for-native-gems branch from 1fe835a to 08b4a29 Compare December 19, 2025 16:43
jdx and others added 3 commits December 19, 2025 11:05
…pilation

The previous commit included headers but not the static libraries (.a files)
or pkg-config files needed by native gem extensions like openssl and psych.

Changes:
- Copy static libraries (libssl.a, libcrypto.a, libyaml.a, etc.) to lib/
- Copy and patch pkg-config files to use ${pcfiledir} for relocatable paths
- Set PKG_CONFIG_PATH in tests so gem install can find the bundled .pc files

The ${pcfiledir} variable in pkg-config expands to the directory containing
the .pc file, allowing the paths to work after the tarball is relocated.

Fixes: jdx/mise#7268 (comment)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous approach tried to match the exact prefix path, but the
path format varies (opt symlink vs cellar path). Using a regex to
match any prefix= line is more robust.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Moves the duplicated code for copying headers, static libraries, and
pkg-config files into a shared helper method in PortableFormulaMixin.

This reduces code duplication across jdx-ruby.rb, jdx-ruby-34.rb,
jdx-ruby-33.rb, and jdx-ruby-32.rb.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
jdx and others added 2 commits December 19, 2025 11:29
libxcrypt headers and pkg-config files were missing from the
portable_deps list, even though libcrypt.a was being copied separately.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The separate `cp libxcrypt.lib/"libcrypt.a"` was causing a permission
error because copy_portable_deps_for_native_gems also copies all *.a
files from libxcrypt. Since libxcrypt is now in portable_deps, the
separate copy is no longer needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
cp_r Dir["#{prefix}/*"], testpath
ENV["PATH"] = "/usr/bin:/bin"
# Set PKG_CONFIG_PATH so gem install can find our bundled pkg-config files
ENV["PKG_CONFIG_PATH"] = "#{testpath}/lib/pkgconfig"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Missing native gem tests for Ruby 3.2 and 3.3

The PKG_CONFIG_PATH environment variable is set in tests for jdx-ruby-32.rb and jdx-ruby-33.rb, but unlike jdx-ruby.rb and jdx-ruby-34.rb, these files don't include the gem install openssl and gem install psych tests that verify the new header-copying feature works. The feature is implemented identically across all versions, but only tested on Ruby 3.4 and HEAD. If there's a version-specific issue with native gem compilation on Ruby 3.2 or 3.3, the tests wouldn't catch it.

Additional Locations (1)

Fix in Cursor Fix in Web

@jdx jdx merged commit 442932e into main Dec 19, 2025
8 checks passed
@jdx jdx deleted the feat/include-headers-for-native-gems branch December 19, 2025 17:48
jdx added a commit to jdx/mise that referenced this pull request Dec 19, 2025
## Summary

When the rubygems_plugin is loaded, prepend the Ruby installation's
`lib/pkgconfig` directory to `PKG_CONFIG_PATH`. This allows gems with
native extensions (openssl, psych, ffi, etc.) to find the bundled
headers and libraries from precompiled Ruby binaries.

## Background

Precompiled Ruby binaries from jdx/ruby now include:
- Headers from portable dependencies (openssl, libyaml, libffi, zlib)
- Static libraries (libssl.a, libcrypto.a, libyaml.a, etc.)
- pkg-config files with relocatable paths using `${pcfiledir}`

However, for `gem install openssl` to work, pkg-config needs to find
these `.pc` files. This PR sets `PKG_CONFIG_PATH` in the rubygems plugin
so it's only active during gem operations.

Related PR: jdx/ruby#1

## Test plan

1. Install precompiled Ruby via mise (experimental mode)
2. Run `gem install openssl` - should succeed without manual
PKG_CONFIG_PATH setup
3. Run `gem install psych` - should find libyaml headers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Prepends Ruby’s `lib/pkgconfig` to `PKG_CONFIG_PATH` in the rubygems
plugin so native gem extensions can find bundled headers and libraries.
> 
> - **Core RubyGems plugin**:
> - Prepend Ruby’s `lib/pkgconfig` to `PKG_CONFIG_PATH` in
`src/plugins/core/assets/rubygems_plugin.rb` (when present) so native
gem extensions can locate bundled headers/libs.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
f0e9b93. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant