Skip to content

Support OCI bundles containing multiple independently restorable snapshots #1865

Description

@simongdavies

Problem

hyperlight-js needs two hyperlight snapshots to persist a LoadedJSSandbox:

  1. Loaded state: the JavaScript runtime with user modules and handlers registered, including mutable JavaScript state produced by handle_event() calls.
  2. Runtime-ready baseline: the state captured immediately after loading the JavaScript runtime, before registering user modules or handlers.

JSSandbox::get_loaded_sandbox() carries the runtime-ready baseline into the resulting LoadedJSSandbox. Later, LoadedJSSandbox::unload() restores that baseline to discard registered guest code and mutable module state and return a JSSandbox.

A typical lifecycle is:

let mut sandbox = builder.build()?.load_runtime()?;
sandbox.add_module("counter", module)?;
sandbox.add_handler("handler", handler)?;

let mut loaded = sandbox.get_loaded_sandbox()?;
loaded.handle_event("handler", event, None)?;

let snapshot = loaded.snapshot()?;
snapshot.save(path, "application-ready")?;

// In another process:
let snapshot = Snapshot::load(path, "application-ready")?;
let loaded = builder
    .build_from_snapshot(snapshot)?
    .restore_loaded_sandbox()?;

let sandbox = loaded.unload()?; // Requires the runtime-ready baseline.

hyperlight-js currently saves the loaded state and runtime-ready baseline as independent top-level OCI manifests. The loaded-state config records the baseline manifest digest in application metadata.

That relationship is not visible through OCI descriptors:

  • Copying only the application-ready tag may omit the baseline manifest and config.
  • OCI garbage collection cannot infer that both manifests form one logical artifact.
  • The copied hyperlight-js snapshot cannot be loaded because hyperlight-js eagerly resolves the baseline manifest.
  • If baseline loading became lazy, LoadedJSSandbox::unload() would still fail when the baseline was absent.

Relationship to incremental snapshots

HIP 0003 / #1823 proposes that each incremental snapshot manifest directly reference every memory blob needed to restore that state.

That solves inherited memory-blob reachability for one Hyperlight snapshot. It does not preserve a second independently restorable state.

The loaded-state manifest may reference memory blobs shared with the runtime-ready baseline, but it does not preserve the baseline’s manifest, config, or semantic role. The HIP intentionally does not retain parent snapshot references.

This request concerns bundling multiple independently restorable snapshots, not representing the internal memory layers of one incremental snapshot.

Current limitation

Snapshot::load resolves a tag or digest only among descriptors directly present in the layout’s root index.json.

The selected descriptor must have media type application/vnd.oci.image.manifest.v1+json. A tagged nested image index containing multiple related snapshot manifests is therefore rejected.

Required Capability

Provide a supported way to represent multiple independently restorable Hyperlight snapshots as one strongly linked OCI artifact.

One possible representation is a tagged OCI image index:

index.json
└── application-ready bundle index
    ├── loaded-state manifest
    └── runtime-ready-baseline manifest

Each child descriptor would carry an explicit application-defined role annotation. Standard OCI tooling could then discover and copy every required manifest and blob by traversing the tagged index.

OCI subject is not sufficient because it defines a weak association with a separate DAG rather than containment within one artifact.

Possible API

  1. Native snapshot-bundle save/load APIs.
  2. Bundle loading resolves a tagged image index and selects children by role.
let bundle = SnapshotBundle::load(path, "application-ready")?;
let loaded = bundle.snapshot("loaded-state")?;
let baseline = bundle.snapshot("runtime-ready-baseline")?;

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions