Skip to content

perf(store): avoid decoding full histories for latest versions #86

Description

@morluto

Problem

Latest-version reads and next-version allocation load, decode and Python-sort every historical document for an id. Each subsequent write repeats that history read, despite SQLite's existing (id, version) index.

Measured opportunity

Full VersionedEntityStore APIs, Python 3.12.13, local SQLite, synthetic scalar entities:

Operation Stored versions Current Indexed implementation
Latest get 1,000 1.601 ms 0.00605 ms
Put 1,000 initially 1.553 ms 0.04944 ms

The reproducible benchmark extracts baseline production methods from commit 05b3310, alternates baseline/final sample order, and checks entity and allocated-version equality. Writes grow each history by 84 versions. Profiling 100 latest reads decodes 100,000 documents before versus 100 after; SQLite's query plan uses the existing compound index.

Acceptance

Use the index for typed latest-version reads, preserving custom stores, namespace routing, exact-version reads and concurrent version allocation. Add a reproducible full-API benchmark, prove returned entity/version parity, and demonstrate indexed reads without decoding the complete history.

DynamoDB's range key uses JSON strings whose lexical order is not numeric; a descending limit there is not a correct substitute and is outside this change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions