Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,18 @@
Wheems follows [semantic versioning](https://semver.org). While it's 0.0.x, any release may
change anything, including commands and config formats, without a migration path.

## Unreleased

**Fixed**

- Search needs SQLite full-text search (FTS5), which Node only bundles from 22.16 in the 22
line and from 24; no Node 23 release has it. `engines` is now `^22.16.0 || >=24`, the CLI and
`wheems service install` refuse other versions with a plain message, `wheems doctor` checks
that this Node's SQLite has full-text search, and search says so instead of failing with
"no such module: fts5".
- `npm run build` now marks `dist/cli/main.js` executable, so a `wheems` linked with `npm link`
runs instead of failing with "Permission denied".

## 0.0.1

The first preview: milestones 0 to 2 of the [roadmap](docs/roadmap.md). Not ready for real use.
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ Wheems was formerly called Opinr.
You need:

- macOS (the service runs under launchd; Linux comes later)
- Node.js 22.13 or later
- Node.js 22.16 or later in the 22 line, or 24 or later (not 23: its SQLite has no full-text
search, which Wheems needs)
- Herdr core 0.1.20 or later (`hc`)
- A Slack workspace where you can add apps

Expand All @@ -102,9 +103,10 @@ npm link # puts the wheems command on your PATH
Clone it anywhere except `~/wheems`. That folder is your install: your private config, notes
and knowledge base, kept in a git folder of its own that is never pushed anywhere.

If your default `node` is older than 22.13, use a newer one, such as Homebrew's
If your default `node` isn't one of those (Homebrew's `node` may be 23), use one that is, such as Homebrew's
(`brew install node@22`), and run `/opt/homebrew/opt/node@22/bin/node dist/cli/main.js` in place
of `wheems`. `wheems doctor` says which Node.js it runs on and fails when it's too old.
of `wheems`. `wheems doctor` says which Node.js it runs on, and fails when it's unsupported or its SQLite has no
full-text search.

## Set up

Expand Down
7 changes: 4 additions & 3 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ most of it in Slack's app settings.

You need:

- Node.js 22.13 or later
- Node.js 22.16 or later in the 22 line, or 24 or later (not 23: its SQLite has no full-text
search, which Wheems needs)
- Herdr core 0.1.20 or later (`hc`)
- A Slack workspace where you can add apps

Expand All @@ -25,9 +26,9 @@ npm install
npm run build
```

If your default `node` is older than 22.13, run Wheems with a newer one, such as Homebrew's:
If your default `node` isn't one of those (Homebrew's `node` may be 23), run Wheems with one that is, such as Homebrew's 22:
`brew install node@22`, then `/opt/homebrew/opt/node@22/bin/node /path/to/wheems/dist/cli/main.js`.
`wheems doctor` says which Node.js it is running on.
`wheems doctor` says which Node.js it is running on, and checks its SQLite has full-text search.

## 2. Run setup

Expand Down
4 changes: 3 additions & 1 deletion docs/plans/milestone-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ Then the live test: a day of real use, then the next morning a new thread asking
built-in guidance also goes into every agent's instructions file, so it works whatever skills
support an engine has.
- **Node's SQLite** prints an "experimental" warning. Wheems hides that warning, and the minimum
Node version rises to 22.13, the first version where it works without a flag.
Node version rises to 22.13, the first version where it works without a flag. (Later raised to
22.16, or 24 and up: Node bundles SQLite with full-text search only from those, and Node 23
never does.)
- **Subscription usage.** Each note and filing run uses your Claude subscription. That's why the
filer defaults to Sonnet and runs once per note, with no reviewer firing every few turns.
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
],
"type": "module",
"engines": {
"node": ">=22.13.0"
"node": "^22.16.0 || >=24"
},
"bin": {
"wheems": "dist/cli/main.js"
Expand All @@ -48,7 +48,7 @@
"CHANGELOG.md"
],
"scripts": {
"build": "tsc -p tsconfig.build.json",
"build": "tsc -p tsconfig.build.json && tsx scripts/make-executable.ts",
"typecheck": "tsc -p tsconfig.json --noEmit",
"lint": "eslint . && prettier --check .",
"format": "prettier --write .",
Expand Down
22 changes: 22 additions & 0 deletions scripts/executable.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/**
* Marks the package's commands (`bin` in package.json) executable, as npm does on install. A
* build runs this after tsc, which writes files without the bit, so a command linked with
* `npm link` runs rather than failing with "Permission denied".
*/
import { chmod, readFile, stat } from 'node:fs/promises';
import { join } from 'node:path';

export async function makeBinsExecutable(root: string): Promise<string[]> {
const pkg = JSON.parse(await readFile(join(root, 'package.json'), 'utf8')) as {
bin?: string | Record<string, string>;
};
const bins = typeof pkg.bin === 'string' ? [pkg.bin] : Object.values(pkg.bin ?? {});
const done: string[] = [];
for (const bin of bins) {
const file = join(root, bin);
const { mode } = await stat(file);
await chmod(file, mode | 0o111);
done.push(bin);
}
return done;
}
7 changes: 7 additions & 0 deletions scripts/make-executable.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/** Run after the build (npm run build): makes the package's commands executable. */
import { fileURLToPath } from 'node:url';
import { makeBinsExecutable } from './executable.js';

for (const bin of await makeBinsExecutable(fileURLToPath(new URL('..', import.meta.url)))) {
console.log(`${bin} is executable`);
}
10 changes: 3 additions & 7 deletions src/cli/commands/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { realpathSync } from 'node:fs';
import { readFile } from 'node:fs/promises';
import { resolve } from 'node:path';
import type { Command } from 'commander';
import { NODE_MINIMUM_VERSION } from '../../doctor/doctor.js';
import { nodeVersionProblem } from '../../util/node-support.js';
import { defaultHome, paths, type Paths } from '../../home.js';
import { request } from '../../service/control.js';
import {
Expand All @@ -13,7 +13,6 @@ import {
uninstallService,
} from '../../service/launchd.js';
import { runService } from '../../service/service.js';
import { isAtLeast } from '../../util/version.js';
import type { Io } from '../program.js';
import { packageVersion } from '../version.js';

Expand All @@ -24,11 +23,8 @@ export function cliPath(): string {

/** Installs the service under launchd with this Node.js and build, and waits for it to answer. */
export async function installAndWait(where: Paths): Promise<boolean> {
if (!isAtLeast(process.versions.node, NODE_MINIMUM_VERSION)) {
throw new Error(
`The service needs Node.js ${NODE_MINIMUM_VERSION} or later, and this is ${process.versions.node}.`,
);
}
const problem = nodeVersionProblem(process.versions.node);
if (problem !== null) throw new Error(problem);
const cli = cliPath();
if (!cli.endsWith('.js')) {
throw new Error('Install the service from a build of Wheems (npm run build), not from source.');
Expand Down
14 changes: 7 additions & 7 deletions src/cli/main.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#!/usr/bin/env node
// Check the Node.js version before loading anything that needs a newer one, so an old
// Node.js gets a plain message rather than a stack trace from a dependency.
const [major = 0, minor = 0] = process.versions.node.split('.').map(Number);
if (major < 22 || (major === 22 && minor < 13)) {
process.stderr.write(
`Wheems needs Node.js 22.13 or later, and this is ${process.versions.node} (${process.execPath}).\n`,
);
// Check the Node.js version before loading anything that needs a newer one, so an
// unsupported Node.js gets a plain message rather than a stack trace from a dependency.
import { nodeVersionProblem } from '../util/node-support.js';

const problem = nodeVersionProblem(process.versions.node);
if (problem !== null) {
process.stderr.write(`${problem}\nThis is ${process.execPath}.\n`);
process.exit(1);
}

Expand Down
33 changes: 19 additions & 14 deletions src/doctor/doctor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ import { HERDR_MINIMUM_VERSION } from '../drivers/runtime/herdr/herdr-runtime.js
import { KnowledgeSearch, type IndexStatus } from '../knowledge/search.js';
import { listAboutYou } from '../review/review.js';
import { listProposals } from '../skills/proposals.js';
import { compareVersions, isAtLeast } from '../util/version.js';
import { compareVersions } from '../util/version.js';
import { NODE_SUPPORTED, nodeVersionProblem } from '../util/node-support.js';
import { fullTextSearchProblem } from '../util/sqlite.js';
export { defaultHome } from '../home.js';

export const NODE_MINIMUM_VERSION = '22.13.0';

export interface DoctorOptions {
/** The install folder. */
home: string;
Expand All @@ -34,6 +34,8 @@ export interface DoctorOptions {
platform?: NodeJS.Platform;
/** Whether an engine trusts a folder: true, false, or null when it can't tell. */
trust?: (engine: string, dir: string) => Promise<boolean | null>;
/** Why SQLite can't do full-text search, or null when it can. */
fullTextSearch?: () => Promise<string | null>;
/** Whether the service is installed and answering. */
service?: (home: string) => Promise<{ installed: boolean; answering: boolean }>;
}
Expand All @@ -51,7 +53,7 @@ export interface DoctorReport {
/** Runs every health check: the system, the runtime, and the install's config. */
export async function runDoctor(options: DoctorOptions): Promise<DoctorReport> {
const sections: DoctorSection[] = [];
sections.push({ title: 'System', checks: systemChecks(options) });
sections.push({ title: 'System', checks: await systemChecks(options) });

const installFile = join(options.home, 'install.yaml');
const hasInstall = await exists(installFile);
Expand Down Expand Up @@ -361,22 +363,25 @@ async function probeService(home: string): Promise<{ installed: boolean; answeri
return { installed: status.loaded, answering };
}

function systemChecks(options: DoctorOptions): HealthCheck[] {
async function systemChecks(options: DoctorOptions): Promise<HealthCheck[]> {
const nodeVersion = options.nodeVersion ?? process.versions.node;
const platform = options.platform ?? process.platform;
const checks: HealthCheck[] = [];
const problem = nodeVersionProblem(nodeVersion);
checks.push(
problem === null
? { name: 'Node.js', status: 'pass', message: `${nodeVersion} is supported` }
: { name: 'Node.js', status: 'fail', message: problem, hint: `Install ${NODE_SUPPORTED}.` },
);
const search = await (options.fullTextSearch ?? (() => fullTextSearchProblem()))();
checks.push(
isAtLeast(nodeVersion, NODE_MINIMUM_VERSION)
? {
name: 'Node.js',
status: 'pass',
message: `${nodeVersion} meets the minimum ${NODE_MINIMUM_VERSION}`,
}
search === null
? { name: 'SQLite search', status: 'pass', message: "Node's SQLite has full-text search" }
: {
name: 'Node.js',
name: 'SQLite search',
status: 'fail',
message: `${nodeVersion} is older than the minimum ${NODE_MINIMUM_VERSION}`,
hint: `Install Node.js ${NODE_MINIMUM_VERSION} or later.`,
message: search,
hint: `Install ${NODE_SUPPORTED}.`,
},
);
checks.push(
Expand Down
3 changes: 2 additions & 1 deletion src/drivers/vector-store/sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
type VectorMatch,
type VectorStoreDriver,
} from '../../contracts/vector-store.js';
import { NODE_SUPPORTED } from '../../util/node-support.js';
import { openDatabase, sqlite } from '../../util/sqlite.js';

export interface SqliteVectorStoreOptions {
Expand Down Expand Up @@ -54,7 +55,7 @@ export class SqliteVectorStore implements VectorStoreDriver {
name: 'vector store',
status: 'fail',
message: `Node's SQLite isn't available: ${error instanceof Error ? error.message : String(error)}`,
hint: 'Use Node.js 22.13 or later.',
hint: `Use ${NODE_SUPPORTED}.`,
},
];
}
Expand Down
9 changes: 8 additions & 1 deletion src/knowledge/search.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import type { InstallConfig } from '../config/schemas.js';
import type { EmbeddingsDriver } from '../contracts/embeddings.js';
import type { DriverRegistry } from '../contracts/registry.js';
import type { VectorCollection, VectorStoreDriver } from '../contracts/vector-store.js';
import { openDatabase } from '../util/sqlite.js';
import { openDatabase, requireFullTextSearch } from '../util/sqlite.js';
import { knowledgePaths, looksLikeSecret, noteTitle } from './notes.js';

/**
Expand Down Expand Up @@ -90,6 +90,11 @@ export interface SearchIndexOptions {
/** Where vectors go; needed only with embeddings. */
vectors: VectorStoreDriver | null;
log?: (message: string) => void;
/**
* Throws when SQLite can't do full-text search, which the index needs. Defaults to checking
* this Node's `node:sqlite`; tests pass their own.
*/
requireFullTextSearch?: () => Promise<void>;
}

interface Current {
Expand Down Expand Up @@ -427,6 +432,7 @@ export class SearchIndex {
`The ${this.workspace} workspace's search index is missing. The service rebuilds it, or run: wheems kb reindex`,
);
}
await (this.options.requireFullTextSearch ?? requireFullTextSearch)();
const db = await openDatabase(this.dbFile(current.generation));
createSchema(db);
const collection =
Expand All @@ -452,6 +458,7 @@ export class SearchIndex {
const vectors = embeddings !== null ? this.options.vectors : null;
if (vectors) await vectors.drop(this.collectionName(generation));

await (this.options.requireFullTextSearch ?? requireFullTextSearch)();
const db = await openDatabase(file);
createSchema(db);
const connection: Connection = {
Expand Down
21 changes: 21 additions & 0 deletions src/util/node-support.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/**
* Which Node.js versions Wheems runs on. Search needs SQLite's full-text search (FTS5) in
* Node's built-in `node:sqlite`. Node bundles SQLite with FTS5 from 22.16.0 and 24.0.0 (nodejs
* PR 57621); no Node 23 release has it, and below 22.13 `node:sqlite` needs a flag. Builds that
* use the system's SQLite can still lack it, so doctor and search also check at run time.
*
* Kept free of imports: the CLI checks this before loading anything else.
*/
export const NODE_SUPPORTED = 'Node.js 22.16 or later in the 22 line, or 24 or later';

/** Why this Node.js version can't run Wheems, or null when it can. */
export function nodeVersionProblem(version: string): string | null {
const [major = 0, minor = 0] = version.replace(/^v/, '').split('.').map(Number);
if (major === 23) {
return `Node.js ${version} can't run Wheems: no Node 23 release has SQLite full-text search, which Wheems's search needs. Use ${NODE_SUPPORTED}.`;
}
if (major < 22 || (major === 22 && minor < 16)) {
return `Node.js ${version} is too old for Wheems. Use ${NODE_SUPPORTED}.`;
}
return null;
}
43 changes: 43 additions & 0 deletions src/util/sqlite.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { DatabaseSync } from 'node:sqlite';
import { NODE_SUPPORTED } from './node-support.js';

type Sqlite = typeof import('node:sqlite');

Expand Down Expand Up @@ -46,3 +47,45 @@ export async function openDatabase(
if (options.readOnly !== true) db.exec('PRAGMA journal_mode = WAL');
return db;
}

/** Something that runs SQL, as `DatabaseSync` does; tests pass a stand-in. */
export interface SqlRunner {
exec(sql: string): void;
close(): void;
}

/**
* Why this SQLite can't do full-text search (FTS5), which search needs, or null when it can.
* `open` makes a scratch database; by default an in-memory one from `node:sqlite`.
*/
export async function fullTextSearchProblem(
open?: () => Promise<SqlRunner>,
): Promise<string | null> {
let db: SqlRunner;
try {
db = open ? await open() : new (await sqlite()).DatabaseSync(':memory:');
} catch (error) {
return `Node's SQLite isn't available: ${describe(error)}. Use ${NODE_SUPPORTED}.`;
}
try {
db.exec('CREATE VIRTUAL TABLE probe USING fts5(text)');
return null;
} catch (error) {
return `This Node.js (${process.versions.node}, ${process.execPath}) has SQLite without full-text search (FTS5), which Wheems's search needs (${describe(error)}). Use ${NODE_SUPPORTED}, from nodejs.org or a package that bundles Node's own SQLite.`;
} finally {
db.close();
}
}

let checked: Promise<string | null> | undefined;

/** Throws a plain explanation when this Node's SQLite has no full-text search. Checks once. */
export async function requireFullTextSearch(): Promise<void> {
checked ??= fullTextSearchProblem();
const problem = await checked;
if (problem !== null) throw new Error(problem);
}

function describe(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
Loading
Loading