Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@
## Important Files

- [index.d.ts](index.d.ts): The spec for this package. When the user updates this spec, the agent should update the native implementation to match
- [README.md.template](README.md.template): editable source for the README wrapper content
- [src/binding.c](src/binding.c): native implementation
- [index.mjs](index.mjs): generated ESM wrapper
- [index.js](index.js): CommonJS loader via `node-gyp-build`
- [scripts/codegen-index-mjs.mts](scripts/codegen-index-mjs.mts): generates `index.mjs` from `index.d.ts`
- [scripts/codegen-index-mjs.mts](scripts/codegen-index-mjs.mts): generates `index.mjs` and the README API section from `index.d.ts`
- [scripts/zig-build-prebuilds.mts](scripts/zig-build-prebuilds.mts): builds and publishes prebuilds
- [test/register-unsafe-pointer-tests.js](test/register-unsafe-pointer-tests.js): shared runtime tests
- [dist-test/index.test.js](dist-test/index.test.js): packed-tarball verification test
Expand All @@ -42,15 +43,17 @@

- `index.d.ts` is the source of truth for the JS-facing API.
- `index.mjs` is generated from `index.d.ts`. Do not hand-edit it.
- `README.md` is generated from `README.md.template` and `index.d.ts`. Do not hand-edit it.
- `bun run build:js` must be stable. Running it should not introduce formatter drift.

## Native/API Sync Rules

- If the public API changes, update all of:
- [index.d.ts](index.d.ts)
- [src/binding.c](src/binding.c)
- [test/register-unsafe-pointer-tests.js](test/register-unsafe-pointer-tests.js)
- [README.md](README.md)
- generated [index.mjs](index.mjs), via `bun run build:js`
- [README.md.template](README.md.template) if the surrounding docs need changes
- generated [index.mjs](index.mjs) and [README.md](README.md), via `bun run build:js`
- Keep the native implementation simple and direct. Avoid duplicating logic, share with helper functions.
- The addon is plain C with Node-API. Keep it small and dependency-free.
- `binding.gyp` exists for source builds. Do not remove the `node-gyp` fallback.
Expand Down Expand Up @@ -101,6 +104,6 @@
- Keep docs dry and concise.
- Use sentence case. Sentences start with a capital letter and end with punctuation.
- Prefer TypeScript in README examples.
- Author the README API section from [index.d.ts](index.d.ts).
- Author the README API section from [index.d.ts](index.d.ts). Edit [README.md.template](README.md.template) for the rest.
- Do not soften the safety language. This package can crash the process, corrupt memory, or enable attacker-controlled execution.
- Preserve existing naming and packaging conventions unless there is a concrete reason to change them.
135 changes: 106 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
<!-- Generated from README.md.template and index.d.ts by scripts/codegen-index-mjs.mts. Do not edit README.md directly. -->
# unsafe-pointer

Unsafely turn `ArrayBuffer` values into raw pointers and raw pointers into
Expand All @@ -11,25 +12,78 @@ especially via FFI like [koffi](https://koffi.dev).

## API

#### unsafePointerOf(buf)

```ts
export function unsafePointerOf<T extends number>(buf: ArrayBufferLike | ArrayBufferView<ArrayBufferLike>): T
```

Returns a pointer to the first byte of the given `ArrayBuffer` or `ArrayBufferView` as a number.

[See Bun's explanation on `number` vs `bigint` pointers](https://bun.com/docs/runtime/ffi#pointers)

```ts
type MyPointerType = number & { __ptr: true }
const ptr: MyPointerType = unsafePointerOf(buffer)
```

- **unsafe**: The JavaScript runtime may move or deallocate objects at will, leading to invalid pointer access. Invalid pointer access can lead to attackers controlling your users' computers.
- **see**: `unsafeBigIntPointerOf` for bigint pointers

#### unsafeBigIntPointerOf(buf)

```ts
export function unsafeBigIntPointerOf<T extends bigint>(buf: ArrayBufferLike | ArrayBufferView<ArrayBufferLike>): T
```

Returns a pointer to the first byte of the given `ArrayBuffer` or `ArrayBufferView` as a bigint.
Bigints are slower than numbers but theoretically a safer way to represent pointers.

[See Bun's explanation on `number` vs `bigint` pointers](https://bun.com/docs/runtime/ffi#pointers)

```ts
type MyPointerType = bigint & { __ptr: true }
const ptr: MyPointerType = unsafeBigIntPointerOf(buffer)
```

- **unsafe**: The JavaScript runtime may move or deallocate objects at will, leading to invalid pointer access. Invalid pointer access can lead to attackers controlling your users' computers.
- **see**: `unsafePointerOf` for number pointers

#### unsafeArrayBufferAt(ptr, offset, byteLength)

```ts
export function unsafeArrayBufferAt<T extends number | bigint>(
ptr: T,
offset: number | undefined,
byteLength: number,
): ArrayBuffer
```

Unsafely create an `ArrayBuffer` aliasing the memory at `pointer + offset` with the given length.

```ts
import {
unsafePointerOf,
unsafeBigIntPointerOf,
unsafeArrayBufferAt,
unsafeCountNonNullBytes,
} from "unsafe-pointer"
type Point3DPointer = number & { __ptr: true, __type: "Point3D" }
type Point3D = Float32Array & { length: 3, __type: "Point3D" }

function UnsafePoint32(ptr: Point3DPointer): Point3D {
return new Float32Array(unsafeArrayBufferAt(ptr, 0, 3 * Float32Array.BYTES_PER_ELEMENT))
}
```

- **unsafe**: Accessing arbitrary memory can lead to attackers controlling your users' computers.

const view = new Uint8Array([1, 2, 3, 4])
#### unsafeCountNonNullBytes(ptr, maxBytes)

const ptr = unsafePointerOf(view)
const bigPtr = unsafeBigIntPointerOf(view)
```ts
export function unsafeCountNonNullBytes<T extends number | bigint>(ptr: T, maxBytes: number): number
```

const alias = new Uint8Array(unsafeArrayBufferAt(ptr, 1, 2))
Iterates from `ptr` until the first null byte is found, or `maxBytes` bytes are reached.
Returns the number of bytes iterated, or `-1` if no null byte found before `maxBytes`.

console.log([...alias]) // [ 2, 3 ]
alias[0] = 99
console.log([...view]) // [ 1, 99, 3, 4 ]
Pass `-1` for `maxBytes` to count all bytes (which like `strlen` is unsafe).

```ts
function unsafeStringAt(ptr: number) {
const length = unsafeCountNonNullBytes(ptr, -1)
return new TextDecoder().decode(new Uint8Array(unsafeArrayBufferAt(ptr, 0, length)))
Expand All @@ -40,20 +94,43 @@ const pointer = unsafePointerOf(cstring)
console.log(unsafeStringAt(pointer)) // "Hello, world!"
```

- `unsafePointerOf<T extends number>(buf: ArrayBuffer | ArrayBufferView): T`
Returns a pointer to the first byte of `buf` as a number.
- `unsafeBigIntPointerOf<T extends bigint>(buf: ArrayBuffer | ArrayBufferView): T`
Returns a pointer to the first byte of `buf` as a bigint.
- `unsafeArrayBufferAt<T extends number | bigint>(ptr: T, offset: number | undefined, byteLength: number): ArrayBuffer`
Creates an `ArrayBuffer` alias for memory at `ptr + offset` with the given `byteLength`.
- `unsafeCountNonNullBytes<T extends number | bigint>(ptr: T, maxBytes: number): number`
Iterates from `ptr` until the first null byte is found, or until `maxBytes`
bytes are reached. Returns the number of bytes iterated, or `-1` if no null
byte is found before `maxBytes`. Pass `-1` for `maxBytes` to count without a
bound.

All APIs are unsafe. The runtime may move or reclaim memory. Invalid pointer
access can lead to crashes, memory corruption, or attacker-controlled execution.
- **unsafe**: Accessing arbitrary memory can lead to attackers controlling your users' computers.

#### type CString

```ts
export type CString<T extends number | bigint> = {
value: string
ptr: T
offset: number
byteLength: number
}
```

A copy of a C UTF-8 string who's properties indicate the original pointer,
offset, and byte length.

As it is a copy, it's safe to use after the pointer is freed.

- **warning**: It is unsafe to assume the c string at `ptr + offset` is still valid
- **warning**: It is unsafe to assume there is a null terminator at `ptr + offset + byteLength`.

#### unsafeCStringAt(ptr, offset, byteLength, result)

```ts
export function unsafeCStringAt<T extends number | bigint, R = Partial<CString<T>>>(
ptr: T,
offset?: number,
byteLength?: number,
result?: R,
): R & CString<T>
```

Copy the C UTF-8 string at `ptr` into a JavaScript string.
If `byteLength` not provided, scans for the closing `\0` character.
Pass a negative `byteLength` to scan at most ABS(byteLength) bytes.

- **unsafe**: Accessing arbitrary memory can lead to attackers controlling your users' computers.

### TypeScript

Expand All @@ -65,7 +142,7 @@ Examples:
```ts
/**
* Emulate bun:ffi interface.
*
*
* https://bun.com/docs/runtime/ffi
*/
import { unsafePointerOf, unsafeArrayBufferAt } from "unsafe-pointer"
Expand Down Expand Up @@ -108,7 +185,7 @@ mise install
bun install
bun run typecheck
bun run rebuild
bun test
bun run test
```

`bun run rebuild` rebuilds the local addon with `node-gyp`.
Expand Down
103 changes: 103 additions & 0 deletions README.md.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
<!-- Generated from README.md.template and index.d.ts by scripts/codegen-index-mjs.mts. Do not edit README.md directly. -->
# unsafe-pointer

Unsafely turn `ArrayBuffer` values into raw pointers and raw pointers into
`ArrayBuffer` aliases.

Warning: invalid pointer use can crash the process, corrupt memory, or let
attackers control the machine. Using this package voids your warranty.

May be useful for interacting with native libraries or doing light witchcraft,
especially via FFI like [koffi](https://koffi.dev).

## API

__APIDOCS__

### TypeScript

You may use `number` or `bigint` branded sub-types of your choosing to represent
pointers. This can be useful for type safety (not! runtime! safety!).

Examples:

```ts
/**
* Emulate bun:ffi interface.
*
* https://bun.com/docs/runtime/ffi
*/
import { unsafePointerOf, unsafeArrayBufferAt } from "unsafe-pointer"
export type Pointer = number & { __pointer__: null }
export const ptr = unsafePointerOf<Pointer>
export const toArrayBuffer = unsafeArrayBufferAt<Pointer>
```

```ts
/**
* Do fancy generics.
*/
import * as unsafe from "unsafe-pointer"
const PointerBrand = Symbol("Pointer")
type Pointer<T = unknown> = number & { [PointerBrand]: true, __type__: T }

export function unsafePointerOf<T extends ArrayBufferView>(view: T): Pointer<T> {
return unsafe.unsafePointerOf(view)
}

type TypedArrayConstructor<T extends ArrayBufferView> = {
new (buffer: ArrayBuffer, byteOffset: number, length: number): T
BYTES_PER_ELEMENT: number
}

export function unsafeTypedArrayAt<T extends ArrayBufferView>(TypedArray: TypedArrayConstructor<T>, ptr: Pointer<T>, length: number): T {
const arrayBuffer = unsafe.unsafeArrayBufferAt(ptr, 0, length * TypedArray.BYTES_PER_ELEMENT)
return new TypedArray(arrayBuffer, 0, length)
}
```

## Development

```sh
# Install tools.
curl https://mise.run | sh
mise install

# Develop.
bun install
bun run typecheck
bun run rebuild
bun run test
```

`bun run rebuild` rebuilds the local addon with `node-gyp`.

## Prebuilds

```sh
bun run build
```

`bun run build` builds the full prebuild matrix with `zig-build` and stages
artifacts under `build/zig-build/` before copying them into `prebuilds/`.

To build a subset of targets:

```sh
./scripts/zig-build-prebuilds.mts darwin-arm64 linux-x64-glibc win32-x64
```

Supported targets:

- `darwin-x64`
- `darwin-arm64`
- `linux-x64-glibc`
- `linux-arm64-glibc`
- `linux-x64-musl`
- `linux-arm64-musl`
- `win32-x64`
- `win32-arm64`

At runtime, `node-gyp-build` loads the matching prebuild from `prebuilds/` when
available. If no matching prebuild is present, source builds via `node-gyp`
should work.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,6 @@
"zig-build": "github:solarwinds/zig-build#fa7428c0a607e4075172346e4d22f7a19ba68fe0"
},
"engines": {
"node": ">=18"
"node": ">=22"
}
}
Loading