Skip to content

perf: defer eager type/streaming imports at import openai - #3843

Open
harsheet-shah wants to merge 1 commit into
openai:mainfrom
harsheet-shah:harsheet/lazy-top-level-imports
Open

perf: defer eager type/streaming imports at import openai#3843
harsheet-shah wants to merge 1 commit into
openai:mainfrom
harsheet-shah:harsheet/lazy-top-level-imports

Conversation

@harsheet-shah

Copy link
Copy Markdown

Summary

import openai eagerly imports the entire Assistants beta type tree and the streaming helpers, building hundreds of pydantic models that most callers never use. This dominates cold-start / first-import latency in serverless and hosted-agent environments (e.g. containers that boot, import the SDK, then serve a single Responses/Chat request).

This PR makes the top-level openai package expose the rarely-needed surfaces lazily while keeping the public API and static typing identical:

  • openai.types is now a LazyProxy (mirrors the existing openai.resources proxy) instead of from . import types.
  • pydantic_function_tool, AssistantEventHandler, AsyncAssistantEventHandler, and the ReconnectingEvent / ReconnectingOverrides websocket types resolve through a module-level __getattr__ (PEP 562).
  • openai.lib defers _tools / _parsing the same way.
  • All names remain statically importable via TYPE_CHECKING re-exports, so IDEs / pyright are unaffected, and from openai import * is unchanged.

Azure / Bedrock module-client support and every existing public object are untouched.

Measurements

openai 3.11.0, warm best-of-3, import openai on top of an already-imported pydantic + httpx2:

import openai eager openai.* submodules
main (eager) ~1,944 ms 835
this PR (lazy) ~1,101 ms 497

~43% faster first import; the Assistants beta subtree (309 modules) is no longer built at import time. It loads transparently on first access (openai.beta, openai.types.beta, etc.).

Tests

Adds tests/test_lazy_imports.py:

  • import openai no longer eagerly loads openai.types.beta or openai.lib.streaming.
  • The lazy top-level exports resolve to the same objects as their canonical modules.
  • The types proxy loads the real module on first attribute access.

Existing tests/test_module_client.py and tests/lib/test_old_api.py continue to pass, and a full import smoke over every openai.types.* subtree + from openai import * verifies no exported name regressed.

`import openai` eagerly imported the entire Assistants `beta` type tree and
the streaming helpers, building hundreds of pydantic models that most callers
never use. This dominates cold-start / first-import latency in serverless and
hosted-agent environments.

Make the top-level `openai` package expose `types`, `pydantic_function_tool`,
`AssistantEventHandler`, `AsyncAssistantEventHandler` and the
`ReconnectingEvent`/`ReconnectingOverrides` websocket types lazily via a
module-level `__getattr__` (PEP 562) plus a `types` LazyProxy, and defer
`openai.lib` helper imports. Public attribute access, `from openai import *`,
and static type checking (via `TYPE_CHECKING` re-exports) are unchanged.

Measured (openai 3.11.0, warm best-of-3): `import openai` drops from ~1,944 ms
to ~1,101 ms (~43%), and eagerly-loaded `openai.*` submodules drop from 835 to
497 -- the Assistants `beta` subtree (309 modules) is no longer built at import.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93f793d2-1b86-4677-908c-3722d4fef290
@harsheet-shah
harsheet-shah requested a review from a team as a code owner September 10, 2026 09:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd68d7aad5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/openai/__init__.py

@eavanvalkenburg eavanvalkenburg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would definitely improve import performance across the board, and complements a fix I made in our package: microsoft/agent-framework#8219

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants