Skip to content

Make litellm an optional extra (basic-memory[litellm]) #1763

Open
Open
Make litellm an optional extra (basic-memory[litellm])#1763
Labels
enhancementNew feature or request

Description

@phernandez
phernandez
opened on Oct 13, 2026

Why

litellm>=1.60.0,<1.92.0 is a core dependency, so every install pulls it in, including environments that never use the litellm embedding or rerank providers (basic-memory-cloud uses fastembed).

litellm constrains openai itself: every release from 1.84.0 through the latest (1.105.0) requires openai>=2.20.0,<3.0.0. Anything installed alongside Basic Memory that needs openai 3 forces the resolver back to litellm 1.83.0, the last release with no upper bound. Pydantic AI 2.5x requires openai>=3.19 for its OpenAI extra, so upgrading Cloud's agent runtime to the latest Pydantic AI (basic-memory-cloud#2470) downgraded litellm from 1.90.2 to 1.83.0.

Removing our <1.92.0 cap would not help, because litellm's own openai cap remains. That cap exists because litellm 1.92 added a Rust bridge whose PyO3 version rejects Python 3.14 (#1051).

Proposal

  • Move litellm from dependencies to [project.optional-dependencies] litellm = ["litellm>=1.60.0,<1.92.0"].
  • The import is already lazy (litellm_provider._import_litellm(), used by the embedding and rerank providers). When the package is missing, raise a clear error naming the setting and the fix: pip install "basic-memory[litellm]" / uv tool install "basic-memory[litellm]".
  • Add a test that the litellm provider raises that error when litellm is not importable, and that fastembed/default paths never import it.
  • Docs: semantic search provider pages and the CHANGELOG note that litellm users add the extra.
  • Check whether core's direct openai>=1.100.2 dependency is still needed, or belongs in the same extra.

After it lands

basic-memory-cloud re-pins core and drops litellm from its lockfile.

Activity

  1. claude commented on Oct 13, 2026

    @claude
    claudeBot
    on Oct 13, 2026 – with Claude
    Contributor

    Claude finished @phernandez's task in 17s —— View job

    Triage

    • Type: Enhancement (packaging change: move litellm from core dependencies to an optional extra)
    • Priority: Medium. It unblocks basic-memory-cloud from the litellm 1.83.0 downgrade forced by Pydantic AI's openai>=3.19 requirement. Nothing is broken for local users.
    • Component: Packaging and dependencies. It touches the embedding and rerank providers, tests and docs. It is not a Cloud-behavior issue, but Cloud is the main beneficiary.
    • Complexity: Simple to Medium. The import is already lazy through _import_litellm(), so the work is:
      • the pyproject.toml change
      • a clear missing-package error
      • tests
      • docs and CHANGELOG
      • a decision on the openai dependency

    Notes

    • Duplicates: none found.
    • Breaking change: existing users of the litellm embedding or rerank providers will need to install the extra after upgrading. The error message and the CHANGELOG should say so plainly.
    • openai>=1.100.2: the audit is the open question. Check whether anything outside the litellm provider imports openai directly. If nothing does, it likely belongs in the same extra, since it would otherwise keep constraining resolvers.
    • Lockfile and dev extras: the dev and test environments need litellm available, so the litellm provider tests keep running. The "not importable" test should simulate the missing package rather than depend on it being absent.
    • Gap in the proposal: the "fastembed/default paths never import litellm" test should also cover any module-level imports and config validation. Today, selecting litellm as a provider in config should fail at provider construction with the actionable error, not at some later call.

    Triage only, so I haven't started on a fix. Mention @claude again with an explicit implementation request if you'd like one.

  2. claude
    added
    enhancementNew feature or request
    on Oct 13, 2026

  3. hyverysmart commented on Oct 17, 2026

    @hyverysmart
    hyverysmart
    on Oct 17, 2026

    Hi @phernandez, I'd like to work on this issue.

    I'm planning to move LiteLLM to an optional extra, improve the missing-dependency error messages, and add regression tests to ensure the default and fastembed providers work without LiteLLM.

    I'll also review the direct OpenAI dependency and update the relevant documentation and changelog.

    Is this issue still available for contribution? I'd appreciate any guidance on the expected scope.

    Thanks!

  4. phernandez commented on Oct 17, 2026

    @phernandez
    phernandez
    on Oct 17, 2026
    MemberAuthor

    @hyverysmart sure. thanks. go 4 it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions