Skip to content

test(adcp): docstring/env-var consistency test#440

Merged
bokelley merged 2 commits into
mainfrom
claude/issue-424-docstring-env-var-consistency
May 3, 2026
Merged

test(adcp): docstring/env-var consistency test#440
bokelley merged 2 commits into
mainfrom
claude/issue-424-docstring-env-var-consistency

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented May 3, 2026

Closes #424

Adds tests/test_docstring_consistency.py. The test AST-parses all public method, class, and module docstrings in src/adcp/, extracts env-var names documented as active, and asserts each one is backed by a real os.environ read in the source tree. CI fails on drift — the original trigger was client.py:381-388 claiming PYTHON_ENV / ENV / ENVIRONMENT were honored when only ADCP_ENV was.

Detection logic:

  • Pattern AVARNAME=value inside RST double-backtick literals (e.g. ADCP_VALIDATION_MODE=strict|warn|off ). Direct signal that a var is documented as settable.
  • Pattern B — SCREAMING_CASE_WITH_UNDERSCORE inside a backtick span, with an env-var context keyword (environ, env var, environment variable, flips) in the ±1-line window. The sliding window catches ADCP_ENV on one line when flips … appears on the next. Restricted to backtick-wrapped tokens; overrides excluded as standalone keyword to prevent OOP-override false positives.
  • Exclusion — lines containing deliberately or not honored are skipped.

What was tested:

  • pytest tests/test_docstring_consistency.py — 1 passed
  • mypy tests/test_docstring_consistency.py — no issues
  • ruff check tests/test_docstring_consistency.py — all checks passed

Pre-PR review:

  • code-reviewer: approved — no blockers; nits surfaced (raw-text scan limitation, PORT coverage gap, SyntaxError comment) all documented in module docstring or addressed in the follow-up commit
  • dx-expert: approved after fixes — initial blockers (greedy regex suffix match, missing ±1 window, bare overrides keyword, bare ignored? exclusion) all resolved in the second commit

Known limitations (documented in module docstring):

  • Single-token env vars without underscores (PORT) are invisible to both patterns
  • _env_vars_in_use() scans raw text; a docstring example embedding a fictional os.environ["FAKE_VAR"] would suppress that name in the in-use set
  • ast.walk collects public methods inside private classes (over-approximation)

Triage-managed PR. This bot does not currently iterate on
review comments or PR conversation threads (only on the source
issue). To unblock:

  • Push fixup commits directly: gh pr checkout <num>
    fix → push.
  • Or re-trigger: comment /triage execute on the source
    issue.

See adcp#3121
for context.

Session: https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}


Generated by Claude Code

claude added 2 commits May 3, 2026 02:52
Adds tests/test_docstring_consistency.py, which AST-parses public method
docstrings across src/adcp/, extracts env-var names documented as active
(via ``VARNAME=value`` assignment syntax or env-var context keywords), and
asserts each one is backed by a real os.environ read in the source tree.

Closes #424

https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}
Address pre-PR review blockers:
- Replace single greedy regex with two-step approach (backtick span
  extraction + word-boundary SCREAMING_CASE search) to avoid matching
  suffixes like P_ENV from ADCP_ENV.
- Add ±1 line sliding window for Pattern B context keywords so
  ADCP_ENV on one line / "flips" on the next is correctly detected.
- Remove standalone "overrides" from context keywords to prevent false
  positives from OOP-override prose.
- Drop bare `ignored?` from exclusion pattern; keep only "deliberately"
  and "not honored" to avoid suppressing legitimate active-var checks.
- Fix module-level lineno fallback to 1 (not 0).
- Add known-limitations section to module docstring.

https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}
@bokelley bokelley marked this pull request as ready for review May 3, 2026 03:59
@bokelley bokelley merged commit d897b62 into main May 3, 2026
11 of 13 checks passed
@bokelley bokelley deleted the claude/issue-424-docstring-env-var-consistency branch May 3, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(testing): docstring-vs-code consistency test

2 participants