diff --git a/docs/README.skills.md b/docs/README.skills.md index e3fd29872..21c0c467d 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -236,6 +236,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [publish-to-pages](../skills/publish-to-pages/SKILL.md) | Publish presentations and web content to GitHub Pages. Converts PPTX, PDF, HTML, or Google Slides to a live GitHub Pages URL. Handles repo creation, file conversion, Pages enablement, and returns the live URL. Use when the user wants to publish, deploy, or share a presentation or HTML file via GitHub Pages. | `scripts/convert-pdf.py`
`scripts/convert-pptx.py`
`scripts/publish.sh` | | [pytest-coverage](../skills/pytest-coverage/SKILL.md) | Run pytest tests with coverage, discover lines missing coverage, and increase coverage to 100%. | None | | [python-mcp-server-generator](../skills/python-mcp-server-generator/SKILL.md) | Generate a complete MCP server project in Python with tools, resources, and proper configuration | None | +| [python-pypi-package-builder](../skills/python-pypi-package-builder/SKILL.md) | End-to-end skill for building, testing, linting, versioning, and publishing a production-grade Python library to PyPI. Covers all four build backends (setuptools+setuptools_scm, hatchling, flit, poetry), PEP 440 versioning, semantic versioning, dynamic git-tag versioning, OOP/SOLID design, type hints (PEP 484/526/544/561), Trusted Publishing (OIDC), and the full PyPA packaging flow. Use for: creating Python packages, pip-installable SDKs, CLI tools, framework plugins, pyproject.toml setup, py.typed, setuptools_scm, semver, mypy, pre-commit, GitHub Actions CI/CD, or PyPI publishing. | `references/architecture-patterns.md`
`references/ci-publishing.md`
`references/community-docs.md`
`references/library-patterns.md`
`references/pyproject-toml.md`
`references/release-governance.md`
`references/testing-quality.md`
`references/tooling-ruff.md`
`references/versioning-strategy.md`
`scripts/scaffold.py` | | [quality-playbook](../skills/quality-playbook/SKILL.md) | Explore any codebase from scratch and generate six quality artifacts: a quality constitution (QUALITY.md), spec-traced functional tests, a code review protocol with regression test generation, an integration testing protocol, a multi-model spec audit (Council of Three), and an AI bootstrap file (AGENTS.md). Includes state machine completeness analysis and missing safeguard detection. Works with any language (Python, Java, Scala, TypeScript, Go, Rust, etc.). Use this skill whenever the user asks to set up a quality playbook, generate functional tests from specifications, create a quality constitution, build testing protocols, audit code against specs, or establish a repeatable quality system for a project. Also trigger when the user mentions 'quality playbook', 'spec audit', 'Council of Three', 'fitness-to-purpose', 'coverage theater', or wants to go beyond basic test generation to build a full quality system grounded in their actual codebase. | `LICENSE.txt`
`references/constitution.md`
`references/defensive_patterns.md`
`references/functional_tests.md`
`references/review_protocols.md`
`references/schema_mapping.md`
`references/spec_audit.md`
`references/verification.md` | | [quasi-coder](../skills/quasi-coder/SKILL.md) | Expert 10x engineer skill for interpreting and implementing code from shorthand, quasi-code, and natural language descriptions. Use when collaborators provide incomplete code snippets, pseudo-code, or descriptions with potential typos or incorrect terminology. Excels at translating non-technical or semi-technical descriptions into production-quality code. | None | | [readme-blueprint-generator](../skills/readme-blueprint-generator/SKILL.md) | Intelligent README.md generation prompt that analyzes project documentation structure and creates comprehensive repository documentation. Scans .github/copilot directory files and copilot-instructions.md to extract project information, technology stack, architecture, development workflow, coding standards, and testing approaches while generating well-structured markdown documentation with proper formatting, cross-references, and developer-focused content. | None | diff --git a/skills/python-pypi-package-builder/SKILL.md b/skills/python-pypi-package-builder/SKILL.md new file mode 100644 index 000000000..e7b4a3841 --- /dev/null +++ b/skills/python-pypi-package-builder/SKILL.md @@ -0,0 +1,444 @@ +--- +name: python-pypi-package-builder +description: 'End-to-end skill for building, testing, linting, versioning, and publishing a production-grade Python library to PyPI. Covers all four build backends (setuptools+setuptools_scm, hatchling, flit, poetry), PEP 440 versioning, semantic versioning, dynamic git-tag versioning, OOP/SOLID design, type hints (PEP 484/526/544/561), Trusted Publishing (OIDC), and the full PyPA packaging flow. Use for: creating Python packages, pip-installable SDKs, CLI tools, framework plugins, pyproject.toml setup, py.typed, setuptools_scm, semver, mypy, pre-commit, GitHub Actions CI/CD, or PyPI publishing.' +--- + +# Python PyPI Package Builder Skill + +A complete, battle-tested guide for building, testing, linting, versioning, typing, and +publishing a production-grade Python library to PyPI — from first commit to community-ready +release. + +> **AI Agent Instruction:** Read this entire file before writing a single line of code or +> creating any file. Every decision — layout, backend, versioning strategy, patterns, CI — +> has a decision rule here. Follow the decision trees in order. This skill applies to any +> Python package type (utility, SDK, CLI, plugin, data library). Do not skip sections. + +--- + +## Quick Navigation + +| Section in this file | What it covers | +|---|---| +| [1. Skill Trigger](#1-skill-trigger) | When to load this skill | +| [2. Package Type Decision](#2-package-type-decision) | Identify what you are building | +| [3. Folder Structure Decision](#3-folder-structure-decision) | src/ vs flat vs monorepo | +| [4. Build Backend Decision](#4-build-backend-decision) | setuptools / hatchling / flit / poetry | +| [5. PyPA Packaging Flow](#5-pypa-packaging-flow) | The canonical publish pipeline | +| [6. Project Structure Templates](#6-project-structure-templates) | Full layouts for every option | +| [7. Versioning Strategy](#7-versioning-strategy) | PEP 440, semver, dynamic vs static | + +| Reference file | What it covers | +|---|---| +| `references/pyproject-toml.md` | All four backend templates, `setuptools_scm`, `py.typed`, tool configs | +| `references/library-patterns.md` | OOP/SOLID, type hints, core class design, factory, protocols, CLI | +| `references/testing-quality.md` | `conftest.py`, unit/backend/async tests, ruff/mypy/pre-commit | +| `references/ci-publishing.md` | `ci.yml`, `publish.yml`, Trusted Publishing, TestPyPI, CHANGELOG, release checklist | +| `references/community-docs.md` | README, docstrings, CONTRIBUTING, SECURITY, anti-patterns, master checklist | +| `references/architecture-patterns.md` | Backend system (plugin/strategy), config layer, transport layer, CLI, backend injection | +| `references/versioning-strategy.md` | PEP 440, SemVer, pre-release, setuptools_scm deep-dive, flit static, decision engine | +| `references/release-governance.md` | Branch strategy, branch protection, OIDC, tag author validation, prevent invalid tags | +| `references/tooling-ruff.md` | Ruff-only setup (replaces black/isort), mypy config, pre-commit, asyncio_mode=auto | + +**Scaffold script:** run `python skills/python-pypi-package-builder/scripts/scaffold.py --name your-package-name` +to generate the entire directory layout, stub files, and `pyproject.toml` in one command. + +--- + +## 1. Skill Trigger + +Load this skill whenever the user wants to: + +- Create, scaffold, or publish a Python package or library to PyPI +- Build a pip-installable SDK, utility, CLI tool, or framework extension +- Set up `pyproject.toml`, linting, mypy, pre-commit, or GitHub Actions for a Python project +- Understand versioning (`setuptools_scm`, PEP 440, semver, static versioning) +- Understand PyPA specs: `py.typed`, `MANIFEST.in`, `RECORD`, classifiers +- Publish to PyPI using Trusted Publishing (OIDC) or API tokens +- Refactor an existing package to follow modern Python packaging standards +- Add type hints, protocols, ABCs, or dataclasses to a Python library +- Apply OOP/SOLID design patterns to a Python package +- Choose between build backends (setuptools, hatchling, flit, poetry) + +**Also trigger for phrases like:** "build a Python SDK", "publish my library", "set up PyPI CI", +"create a pip package", "how do I publish to PyPI", "pyproject.toml help", "PEP 561 typed", +"setuptools_scm version", "semver Python", "PEP 440", "git tag release", "Trusted Publishing". + +--- + +## 2. Package Type Decision + +Identify what the user is building **before** writing any code. Each type has distinct patterns. + +### Decision Table + +| Type | Core Pattern | Entry Point | Key Deps | Example Packages | +|---|---|---|---|---| +| **Utility library** | Module of pure functions + helpers | Import API only | Minimal | `arrow`, `humanize`, `boltons`, `more-itertools` | +| **API client / SDK** | Class with methods, auth, retry logic | Import API only | `httpx` or `requests` | `boto3`, `stripe-python`, `openai` | +| **CLI tool** | Command functions + argument parser | `[project.scripts]` or `[project.entry-points]` | `click` or `typer` | `black`, `ruff`, `httpie`, `rich` | +| **Framework plugin** | Plugin class, hook registration | `[project.entry-points."framework.plugin"]` | Framework dep | `pytest-*`, `django-*`, `flask-*` | +| **Data processing library** | Classes + functional pipeline | Import API only | Optional: `numpy`, `pandas` | `pydantic`, `marshmallow`, `cerberus` | +| **Mixed / generic** | Combination of above | Varies | Varies | Many real-world packages | + +**Decision Rule:** Ask the user if unclear. A package can combine types (e.g., SDK with a CLI +entry point) — use the primary type for structural decisions and add secondary type patterns on top. + +For implementation patterns of each type, see `references/library-patterns.md`. + +### Package Naming Rules + +- PyPI name: all lowercase, hyphens — `my-python-library` +- Python import name: underscores — `my_python_library` +- Check availability: https://pypi.org/search/ before starting +- Avoid shadowing popular packages (verify `pip install ` fails first) + +--- + +## 3. Folder Structure Decision + +### Decision Tree + +``` +Does the package have 5+ internal modules OR multiple contributors OR complex sub-packages? +├── YES → Use src/ layout +│ Reason: prevents accidental import of uninstalled code during development; +│ separates source from project root files; PyPA-recommended for large projects. +│ +├── NO → Is it a single-module, focused package (e.g., one file + helpers)? +│ ├── YES → Use flat layout +│ └── NO (medium complexity) → Use flat layout, migrate to src/ if it grows +│ +└── Is it multiple related packages under one namespace (e.g., myorg.http, myorg.db)? + └── YES → Use namespace/monorepo layout +``` + +### Quick Rule Summary + +| Situation | Use | +|---|---| +| New project, unknown future size | `src/` layout (safest default) | +| Single-purpose, 1–4 modules | Flat layout | +| Large library, many contributors | `src/` layout | +| Multiple packages in one repo | Namespace / monorepo | +| Migrating old flat project | Keep flat; migrate to `src/` at next major version | + +--- + +## 4. Build Backend Decision + +### Decision Tree + +``` +Does the user need version derived automatically from git tags? +├── YES → Use setuptools + setuptools_scm +│ (git tag v1.0.0 → that IS your release workflow) +│ +└── NO → Does the user want an all-in-one tool (deps + build + publish)? + ├── YES → Use poetry (v2+ supports standard [project] table) + │ + └── NO → Is the package pure Python with no C extensions? + ├── YES, minimal config preferred → Use flit + │ (zero config, auto-discovers version from __version__) + │ + └── YES, modern & fast preferred → Use hatchling + (zero-config, plugin system, no setup.py needed) + +Does the package have C/Cython/Fortran extensions? +└── YES → MUST use setuptools (only backend with full native extension support) +``` + +### Backend Comparison + +| Backend | Version source | Config | C extensions | Best for | +|---|---|---|---|---| +| `setuptools` + `setuptools_scm` | git tags (automatic) | `pyproject.toml` + optional `setup.py` shim | Yes | Projects with git-tag releases; any complexity | +| `hatchling` | manual or plugin | `pyproject.toml` only | No | New pure-Python projects; fast, modern | +| `flit` | `__version__` in `__init__.py` | `pyproject.toml` only | No | Very simple, single-module packages | +| `poetry` | `pyproject.toml` field | `pyproject.toml` only | No | Teams wanting integrated dep management | + +For all four complete `pyproject.toml` templates, see `references/pyproject-toml.md`. + +--- + +## 5. PyPA Packaging Flow + +This is the canonical end-to-end flow from source code to user install. +**Every step must be understood before publishing.** + +``` +1. SOURCE TREE + Your code in version control (git) + └── pyproject.toml describes metadata + build system + +2. BUILD + python -m build + └── Produces two artifacts in dist/: + ├── *.tar.gz → source distribution (sdist) + └── *.whl → built distribution (wheel) — preferred by pip + +3. VALIDATE + twine check dist/* + └── Checks metadata, README rendering, and PyPI compatibility + +4. TEST PUBLISH (first release only) + twine upload --repository testpypi dist/* + └── Verify: pip install --index-url https://test.pypi.org/simple/ your-package + +5. PUBLISH + twine upload dist/* ← manual fallback + OR GitHub Actions publish.yml ← recommended (Trusted Publishing / OIDC) + +6. USER INSTALL + pip install your-package + pip install "your-package[extra]" +``` + +### Key PyPA Concepts + +| Concept | What it means | +|---|---| +| **sdist** | Source distribution — your source + metadata; used when no wheel is available | +| **wheel (.whl)** | Pre-built binary — pip extracts directly into site-packages; no build step | +| **PEP 517/518** | Standard build system interface via `pyproject.toml [build-system]` table | +| **PEP 621** | Standard `[project]` table in `pyproject.toml`; all modern backends support it | +| **PEP 639** | `license` key as SPDX string (e.g., `"MIT"`, `"Apache-2.0"`) — not `{text = "MIT"}` | +| **PEP 561** | `py.typed` empty marker file — tells mypy/IDEs this package ships type information | + +For complete CI workflow and publishing setup, see `references/ci-publishing.md`. + +--- + +## 6. Project Structure Templates + +### A. src/ Layout (Recommended default for new projects) + +``` +your-package/ +├── src/ +│ └── your_package/ +│ ├── __init__.py # Public API: __all__, __version__ +│ ├── py.typed # PEP 561 marker — EMPTY FILE +│ ├── core.py # Primary implementation +│ ├── client.py # (API client type) or remove +│ ├── cli.py # (CLI type) click/typer commands, or remove +│ ├── config.py # Settings / configuration dataclass +│ ├── exceptions.py # Custom exception hierarchy +│ ├── models.py # Data classes, Pydantic models, TypedDicts +│ ├── utils.py # Internal helpers (prefix _utils if private) +│ ├── types.py # Shared type aliases and TypeVars +│ └── backends/ # (Plugin pattern) — remove if not needed +│ ├── __init__.py # Protocol / ABC interface definition +│ ├── memory.py # Default zero-dep implementation +│ └── redis.py # Optional heavy implementation +├── tests/ +│ ├── __init__.py +│ ├── conftest.py # Shared fixtures +│ ├── unit/ +│ │ ├── __init__.py +│ │ ├── test_core.py +│ │ ├── test_config.py +│ │ └── test_models.py +│ ├── integration/ +│ │ ├── __init__.py +│ │ └── test_backends.py +│ └── e2e/ # Optional: end-to-end tests +│ └── __init__.py +├── docs/ # Optional: mkdocs or sphinx +├── scripts/ +│ └── scaffold.py +├── .github/ +│ ├── workflows/ +│ │ ├── ci.yml +│ │ └── publish.yml +│ └── ISSUE_TEMPLATE/ +│ ├── bug_report.md +│ └── feature_request.md +├── .pre-commit-config.yaml +├── pyproject.toml +├── CHANGELOG.md +├── CONTRIBUTING.md +├── SECURITY.md +├── LICENSE +├── README.md +└── .gitignore +``` + +### B. Flat Layout (Small / focused packages) + +``` +your-package/ +├── your_package/ # ← at root, not inside src/ +│ ├── __init__.py +│ ├── py.typed +│ └── ... (same internal structure) +├── tests/ +└── ... (same top-level files) +``` + +### C. Namespace / Monorepo Layout (Multiple related packages) + +``` +your-org/ +├── packages/ +│ ├── your-org-core/ +│ │ ├── src/your_org/core/ +│ │ └── pyproject.toml +│ ├── your-org-http/ +│ │ ├── src/your_org/http/ +│ │ └── pyproject.toml +│ └── your-org-cli/ +│ ├── src/your_org/cli/ +│ └── pyproject.toml +├── .github/workflows/ +└── README.md +``` + +Each sub-package has its own `pyproject.toml`. They share the `your_org` namespace via PEP 420 +implicit namespace packages (no `__init__.py` in the namespace root). + +### Internal Module Guidelines + +| File | Purpose | When to include | +|---|---|---| +| `__init__.py` | Public API surface; re-exports; `__version__` | Always | +| `py.typed` | PEP 561 typed-package marker (empty) | Always | +| `core.py` | Primary class / main logic | Always | +| `config.py` | Settings dataclass or Pydantic model | When configurable | +| `exceptions.py` | Exception hierarchy (`YourBaseError` → specifics) | Always | +| `models.py` | Data models / DTOs / TypedDicts | When data-heavy | +| `utils.py` | Internal helpers (not part of public API) | As needed | +| `types.py` | Shared `TypeVar`, `TypeAlias`, `Protocol` definitions | When complex typing | +| `cli.py` | CLI entry points (click/typer) | CLI type only | +| `backends/` | Plugin/strategy pattern | When swappable implementations | +| `_compat.py` | Python version compatibility shims | When 3.9–3.13 compat needed | + +--- + +## 7. Versioning Strategy + +### PEP 440 — The Standard + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +Examples: + 1.0.0 Stable release + 1.0.0a1 Alpha (pre-release) + 1.0.0b2 Beta + 1.0.0rc1 Release candidate + 1.0.0.post1 Post-release (e.g., packaging fix only) + 1.0.0.dev1 Development snapshot (not for PyPI) +``` + +### Semantic Versioning (recommended) + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking API change (remove/rename public function/class/arg) +MINOR: New feature, fully backward-compatible +PATCH: Bug fix, no API change +``` + +### Dynamic versioning with setuptools_scm (recommended for git-tag workflows) + +```bash +# How it works: +git tag v1.0.0 → installed version = 1.0.0 +git tag v1.1.0 → installed version = 1.1.0 +(commits after tag) → version = 1.1.0.post1 (suffix stripped for PyPI) + +# In code — NEVER hardcode when using setuptools_scm: +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Required `pyproject.toml` config: +```toml +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +g from breaking PyPI uploads +``` + +**Critical:** always set `fetch-depth: 0` in every CI checkout step. Without full git history, +`setuptools_scm` cannot find tags and the build version silently falls back to `0.0.0+dev`. + +### Static versioning (flit, hatchling manual, poetry) + +```python +# your_package/__init__.py +__version__ = "1.0.0" # Update this before every release +``` + +### Version specifier best practices for dependencies + +```toml +# In [project] dependencies: +"httpx>=0.24" # Minimum version — PREFERRED for libraries +"httpx>=0.24,<1.0" # Upper bound only when a known breaking change exists +"httpx==0.27.0" # Pin exactly ONLY in applications, NOT libraries + +# NEVER do this in a library — it breaks dependency resolution for users: +# "httpx~=0.24.0" # Too tight +# "httpx==0.27.*" # Fragile +``` + +### Version bump → release flow + +```bash +# 1. Update CHANGELOG.md — move [Unreleased] entries to [x.y.z] - YYYY-MM-DD +# 2. Commit the changelog +git add CHANGELOG.md +git commit -m "chore: prepare release vX.Y.Z" +# 3. Tag and push — this triggers publish.yml automatically +git tag vX.Y.Z +git push origin main --tags +# 4. Monitor GitHub Actions → verify on https://pypi.org/project/your-package/ +``` + +For complete pyproject.toml templates for all four backends, see `references/pyproject-toml.md`. + +--- + +## Where to Go Next + +After understanding decisions and structure: + +1. **Set up `pyproject.toml`** → `references/pyproject-toml.md` + All four backend templates (setuptools+scm, hatchling, flit, poetry), full tool configs, + `py.typed` setup, versioning config. + +2. **Write your library code** → `references/library-patterns.md` + OOP/SOLID principles, type hints (PEP 484/526/544/561), core class design, factory functions, + `__init__.py`, plugin/backend pattern, CLI entry point. + +3. **Add tests and code quality** → `references/testing-quality.md` + `conftest.py`, unit/backend/async tests, parametrize, ruff/mypy/pre-commit setup. + +4. **Set up CI/CD and publish** → `references/ci-publishing.md` + `ci.yml`, `publish.yml` with Trusted Publishing (OIDC, no API tokens), CHANGELOG format, + release checklist. + +5. **Polish for community/OSS** → `references/community-docs.md` + README sections, docstring format, CONTRIBUTING, SECURITY, issue templates, anti-patterns + table, and master release checklist. + +6. **Design backends, config, transport, CLI** → `references/architecture-patterns.md` + Backend system (plugin/strategy pattern), Settings dataclass, HTTP transport layer, + CLI with click/typer, backend injection rules. + +7. **Choose and implement a versioning strategy** → `references/versioning-strategy.md` + PEP 440 canonical forms, SemVer rules, pre-release identifiers, setuptools_scm deep-dive, + flit static versioning, decision engine (DEFAULT/BEGINNER/MINIMAL). + +8. **Govern releases and secure the publish pipeline** → `references/release-governance.md` + Branch strategy, branch protection rules, OIDC Trusted Publishing setup, tag author + validation in CI, tag format enforcement, full governed `publish.yml`. + +9. **Simplify tooling with Ruff** → `references/tooling-ruff.md` + Ruff-only setup replacing black/isort/flake8, mypy config, pre-commit hooks, + asyncio_mode=auto (remove @pytest.mark.asyncio), migration guide. diff --git a/skills/python-pypi-package-builder/references/architecture-patterns.md b/skills/python-pypi-package-builder/references/architecture-patterns.md new file mode 100644 index 000000000..5ed08503e --- /dev/null +++ b/skills/python-pypi-package-builder/references/architecture-patterns.md @@ -0,0 +1,555 @@ +# Architecture Patterns — Backend System, Config, Transport, CLI + +## Table of Contents +1. [Backend System (Plugin/Strategy Pattern)](#1-backend-system-pluginstrategy-pattern) +2. [Config Layer (Settings Dataclass)](#2-config-layer-settings-dataclass) +3. [Transport Layer (HTTP Client Abstraction)](#3-transport-layer-http-client-abstraction) +4. [CLI Support](#4-cli-support) +5. [Backend Injection in Core Client](#5-backend-injection-in-core-client) +6. [Decision Rules](#6-decision-rules) + +--- + +## 1. Backend System (Plugin/Strategy Pattern) + +Structure your `backends/` sub-package with a clear base protocol, a zero-dependency default +implementation, and optional heavy implementations behind extras. + +### Directory Layout + +``` +your_package/ + backends/ + __init__.py # Exports BaseBackend + factory; holds the Protocol/ABC + base.py # Abstract base class (ABC) or Protocol definition + memory.py # Default, zero-dependency in-memory implementation + redis.py # Optional, heavier implementation (guarded by extras) +``` + +### `backends/base.py` — Abstract Interface + +```python +# your_package/backends/base.py +from __future__ import annotations + +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage/processing backend. + + All concrete backends must implement these methods. + Never import heavy dependencies at module level — guard them inside the class. + """ + + @abstractmethod + def get(self, key: str) -> str | None: + """Retrieve a value by key. Return None when the key does not exist.""" + ... + + @abstractmethod + def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value with an optional TTL (seconds).""" + ... + + @abstractmethod + def delete(self, key: str) -> None: + """Remove a key. No-op when the key does not exist.""" + ... + + def close(self) -> None: # noqa: B027 (intentionally non-abstract) + """Optional cleanup hook. Override in backends that hold connections.""" +``` + +### `backends/memory.py` — Default Zero-Dep Implementation + +```python +# your_package/backends/memory.py +from __future__ import annotations + +import time +from collections.abc import Iterator +from contextlib import contextmanager +from threading import Lock + +from .base import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. No external dependencies required.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = Lock() + + def get(self, key: str) -> str | None: + with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.monotonic() > expires_at: + del self._store[key] + return None + return value + + def set(self, key: str, value: str, ttl: int | None = None) -> None: + expires_at = time.monotonic() + ttl if ttl is not None else None + with self._lock: + self._store[key] = (value, expires_at) + + def delete(self, key: str) -> None: + with self._lock: + self._store.pop(key, None) +``` + +### `backends/redis.py` — Optional Heavy Implementation + +```python +# your_package/backends/redis.py +from __future__ import annotations + +from .base import BaseBackend + + +class RedisBackend(BaseBackend): + """Redis-backed implementation. Requires: pip install your-package[redis]""" + + def __init__(self, url: str = "redis://localhost:6379/0") -> None: + try: + import redis as _redis + except ImportError as exc: + raise ImportError( + "RedisBackend requires redis. " + "Install it with: pip install your-package[redis]" + ) from exc + self._client = _redis.from_url(url, decode_responses=True) + + def get(self, key: str) -> str | None: + return self._client.get(key) # type: ignore[return-value] + + def set(self, key: str, value: str, ttl: int | None = None) -> None: + if ttl is not None: + self._client.setex(key, ttl, value) + else: + self._client.set(key, value) + + def delete(self, key: str) -> None: + self._client.delete(key) + + def close(self) -> None: + self._client.close() +``` + +### `backends/__init__.py` — Public API + Factory + +```python +# your_package/backends/__init__.py +from __future__ import annotations + +from .base import BaseBackend +from .memory import MemoryBackend + +__all__ = ["BaseBackend", "MemoryBackend", "get_backend"] + + +def get_backend(backend_type: str = "memory", **kwargs: object) -> BaseBackend: + """Factory: return the requested backend instance. + + Args: + backend_type: "memory" (default) or "redis". + **kwargs: Forwarded to the backend constructor. + """ + if backend_type == "memory": + return MemoryBackend() + if backend_type == "redis": + from .redis import RedisBackend # Late import — redis is optional + return RedisBackend(**kwargs) # type: ignore[arg-type] + raise ValueError(f"Unknown backend type: {backend_type!r}") +``` + +--- + +## 2. Config Layer (Settings Dataclass) + +Centralise all configuration in one `config.py` module. Avoid scattering magic values and +`os.environ` calls across the codebase. + +### `config.py` + +```python +# your_package/config.py +from __future__ import annotations + +import os +from dataclasses import dataclass, field + + +@dataclass +class Settings: + """All runtime configuration for your package. + + Attributes: + api_key: Authentication credential. Never log or expose this. + timeout: HTTP request timeout in seconds. + retries: Maximum number of retry attempts on transient failures. + base_url: API base URL. Override in tests with a local server. + """ + + api_key: str + timeout: int = 30 + retries: int = 3 + base_url: str = "https://api.example.com/v1" + + def __post_init__(self) -> None: + if not self.api_key: + raise ValueError("api_key must not be empty") + if self.timeout < 1: + raise ValueError("timeout must be >= 1") + if self.retries < 0: + raise ValueError("retries must be >= 0") + + @classmethod + def from_env(cls) -> "Settings": + """Construct Settings from environment variables. + + Required env var: YOUR_PACKAGE_API_KEY + Optional env vars: YOUR_PACKAGE_TIMEOUT, YOUR_PACKAGE_RETRIES + """ + api_key = os.environ.get("YOUR_PACKAGE_API_KEY", "") + timeout = int(os.environ.get("YOUR_PACKAGE_TIMEOUT", "30")) + retries = int(os.environ.get("YOUR_PACKAGE_RETRIES", "3")) + return cls(api_key=api_key, timeout=timeout, retries=retries) +``` + +### Using Pydantic (optional, for larger projects) + +```python +# your_package/config.py — Pydantic v2 variant +from __future__ import annotations + +from pydantic import Field +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + api_key: str = Field(..., min_length=1) + timeout: int = Field(30, ge=1) + retries: int = Field(3, ge=0) + base_url: str = "https://api.example.com/v1" + + model_config = {"env_prefix": "YOUR_PACKAGE_"} +``` + +--- + +## 3. Transport Layer (HTTP Client Abstraction) + +Isolate all HTTP concerns — headers, retries, timeouts, error parsing — in a dedicated +`transport/` sub-package. The core client depends on the transport abstraction, not on `httpx` +or `requests` directly. + +### Directory Layout + +``` +your_package/ + transport/ + __init__.py # Re-exports HttpTransport + http.py # Concrete httpx-based transport +``` + +### `transport/http.py` + +```python +# your_package/transport/http.py +from __future__ import annotations + +from typing import Any + +import httpx + +from ..config import Settings +from ..exceptions import YourPackageError, RateLimitError, AuthenticationError + + +class HttpTransport: + """Thin httpx wrapper that centralises auth, retries, and error mapping.""" + + def __init__(self, settings: Settings) -> None: + self._settings = settings + self._client = httpx.Client( + base_url=settings.base_url, + timeout=settings.timeout, + headers={"Authorization": f"Bearer {settings.api_key}"}, + ) + + def request( + self, + method: str, + path: str, + *, + json: dict[str, Any] | None = None, + params: dict[str, Any] | None = None, + ) -> dict[str, Any]: + """Send an HTTP request and return the parsed JSON body. + + Raises: + AuthenticationError: on 401. + RateLimitError: on 429. + YourPackageError: on all other non-2xx responses. + """ + response = self._client.request(method, path, json=json, params=params) + self._raise_for_status(response) + return response.json() + + def _raise_for_status(self, response: httpx.Response) -> None: + if response.status_code == 401: + raise AuthenticationError("Invalid or expired API key.") + if response.status_code == 429: + raise RateLimitError("Rate limit exceeded. Back off and retry.") + if response.is_error: + raise YourPackageError( + f"API error {response.status_code}: {response.text[:200]}" + ) + + def close(self) -> None: + self._client.close() + + def __enter__(self) -> "HttpTransport": + return self + + def __exit__(self, *args: object) -> None: + self.close() +``` + +### Async variant + +```python +# your_package/transport/async_http.py +from __future__ import annotations + +from typing import Any + +import httpx + +from ..config import Settings +from ..exceptions import YourPackageError, RateLimitError, AuthenticationError + + +class AsyncHttpTransport: + """Async httpx wrapper. Use with `async with AsyncHttpTransport(...) as t:`.""" + + def __init__(self, settings: Settings) -> None: + self._settings = settings + self._client = httpx.AsyncClient( + base_url=settings.base_url, + timeout=settings.timeout, + headers={"Authorization": f"Bearer {settings.api_key}"}, + ) + + async def request( + self, + method: str, + path: str, + *, + json: dict[str, Any] | None = None, + params: dict[str, Any] | None = None, + ) -> dict[str, Any]: + response = await self._client.request(method, path, json=json, params=params) + self._raise_for_status(response) + return response.json() + + def _raise_for_status(self, response: httpx.Response) -> None: + if response.status_code == 401: + raise AuthenticationError("Invalid or expired API key.") + if response.status_code == 429: + raise RateLimitError("Rate limit exceeded. Back off and retry.") + if response.is_error: + raise YourPackageError( + f"API error {response.status_code}: {response.text[:200]}" + ) + + async def aclose(self) -> None: + await self._client.aclose() + + async def __aenter__(self) -> "AsyncHttpTransport": + return self + + async def __aexit__(self, *args: object) -> None: + await self.aclose() +``` + +--- + +## 4. CLI Support + +Add a CLI entry point via `[project.scripts]` in `pyproject.toml`. + +### `pyproject.toml` entry + +```toml +[project.scripts] +your-cli = "your_package.cli:main" +``` + +After installation, the user can run `your-cli --help` directly from the terminal. + +### `cli.py` — Using Click + +```python +# your_package/cli.py +from __future__ import annotations + +import sys + +import click + +from .config import Settings +from .core import YourClient + + +@click.group() +@click.version_option() +def main() -> None: + """your-package CLI — interact with the API from the command line.""" + + +@main.command() +@click.option("--api-key", envvar="YOUR_PACKAGE_API_KEY", required=True, help="API key.") +@click.option("--timeout", default=30, show_default=True, help="Request timeout (s).") +@click.argument("query") +def search(api_key: str, timeout: int, query: str) -> None: + """Search the API and print results.""" + settings = Settings(api_key=api_key, timeout=timeout) + client = YourClient(settings=settings) + try: + results = client.search(query) + for item in results: + click.echo(item) + except Exception as exc: + click.echo(f"Error: {exc}", err=True) + sys.exit(1) +``` + +### `cli.py` — Using Typer (modern alternative) + +```python +# your_package/cli.py +from __future__ import annotations + +import typer + +from .config import Settings +from .core import YourClient + +app = typer.Typer(help="your-package CLI.") + + +@app.command() +def search( + query: str = typer.Argument(..., help="Search query."), + api_key: str = typer.Option(..., envvar="YOUR_PACKAGE_API_KEY"), + timeout: int = typer.Option(30, help="Request timeout (s)."), +) -> None: + """Search the API and print results.""" + settings = Settings(api_key=api_key, timeout=timeout) + client = YourClient(settings=settings) + results = client.search(query) + for item in results: + typer.echo(item) + + +def main() -> None: + app() +``` + +--- + +## 5. Backend Injection in Core Client + +**Critical:** always accept `backend` as a constructor argument. Never instantiate the backend +inside the constructor without a fallback parameter — that makes testing impossible. + +```python +# your_package/core.py +from __future__ import annotations + +from .backends.base import BaseBackend +from .backends.memory import MemoryBackend +from .config import Settings + + +class YourClient: + """Primary client. Accepts an injected backend for testability. + + Args: + settings: Resolved configuration. Use Settings.from_env() for production. + backend: Storage/processing backend. Defaults to MemoryBackend when None. + timeout: Deprecated — pass a Settings object instead. + retries: Deprecated — pass a Settings object instead. + """ + + def __init__( + self, + api_key: str | None = None, + *, + settings: Settings | None = None, + backend: BaseBackend | None = None, + timeout: int = 30, + retries: int = 3, + ) -> None: + if settings is None: + if api_key is None: + raise ValueError("Provide either 'api_key' or 'settings'.") + settings = Settings(api_key=api_key, timeout=timeout, retries=retries) + self._settings = settings + # CORRECT — default injected, not hardcoded + self.backend: BaseBackend = backend if backend is not None else MemoryBackend() + + # ... methods +``` + +### Anti-Pattern — Never Do This + +```python +# BAD: hardcodes the backend; impossible to swap in tests +class YourClient: + def __init__(self, api_key: str) -> None: + self.backend = MemoryBackend() # ← no injection possible + +# BAD: hardcodes the package name literal in imports +from your_package.backends.memory import MemoryBackend # only fine in your_package itself +# use relative imports inside the package: +from .backends.memory import MemoryBackend # ← correct +``` + +--- + +## 6. Decision Rules + +``` +Does the package interact with external state (cache, DB, queue)? +├── YES → Add backends/ with BaseBackend + MemoryBackend +│ Add optional heavy backends behind extras_require +│ +└── NO → Skip backends/ entirely; keep core.py simple + +Does the package call an external HTTP API? +├── YES → Add transport/http.py; inject via Settings +│ +└── NO → Skip transport/ + +Does the package need a command-line interface? +├── YES, simple (1–3 commands) → Use argparse or click +│ Add [project.scripts] in pyproject.toml +│ +├── YES, complex (sub-commands, plugins) → Use click or typer +│ +└── NO → Skip cli.py + +Does runtime behaviour depend on user-supplied config? +├── YES → Add config.py with Settings dataclass +│ Expose Settings.from_env() for production use +│ +└── NO → Accept params directly in the constructor +``` diff --git a/skills/python-pypi-package-builder/references/ci-publishing.md b/skills/python-pypi-package-builder/references/ci-publishing.md new file mode 100644 index 000000000..f96fc7615 --- /dev/null +++ b/skills/python-pypi-package-builder/references/ci-publishing.md @@ -0,0 +1,315 @@ +# CI/CD, Publishing, and Changelog + +## Table of Contents +1. [Changelog format](#1-changelog-format) +2. [ci.yml — lint, type-check, test matrix](#2-ciyml) +3. [publish.yml — triggered on version tags](#3-publishyml) +4. [PyPI Trusted Publishing (no API tokens)](#4-pypi-trusted-publishing) +5. [Manual publish fallback](#5-manual-publish-fallback) +6. [Release checklist](#6-release-checklist) +7. [Verify py.typed ships in the wheel](#7-verify-pytyped-ships-in-the-wheel) +8. [Semver change-type guide](#8-semver-change-type-guide) + +--- + +## 1. Changelog Format + +Keep a `CHANGELOG.md` following [Keep a Changelog](https://keepachangelog.com/) conventions. +Every PR should update the `[Unreleased]` section. Before releasing, move those entries to a +new version section with the date. + +```markdown +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [Unreleased] + +### Added +- (in-progress features go here) + +--- + +## [1.0.0] - 2026-04-02 + +### Added +- Initial stable release +- `YourMiddleware` with gradual, strict, and combined modes +- In-memory backend (no extra deps) +- Optional Redis backend (`pip install pkg[redis]`) +- Per-route override via `Depends(RouteThrottle(...))` +- `py.typed` marker — PEP 561 typed package +- GitHub Actions CI: lint, mypy, test matrix, Trusted Publishing + +### Changed +### Fixed +### Removed + +--- + +## [0.1.0] - 2026-03-01 + +### Added +- Initial project scaffold + +[Unreleased]: https://github.com/you/your-package/compare/v1.0.0...HEAD +[1.0.0]: https://github.com/you/your-package/compare/v0.1.0...v1.0.0 +[0.1.0]: https://github.com/you/your-package/releases/tag/v0.1.0 +``` + +### Semver — what bumps what + +| Change type | Bump | Example | +|---|---|---| +| Breaking API change | MAJOR | `1.0.0 → 2.0.0` | +| New feature, backward-compatible | MINOR | `1.0.0 → 1.1.0` | +| Bug fix | PATCH | `1.0.0 → 1.0.1` | + +--- + +## 2. `ci.yml` + +Runs on every push and pull request. Tests across all supported Python versions. + +```yaml +# .github/workflows/ci.yml +name: CI + +on: + push: + branches: [main, master] + pull_request: + branches: [main, master] + +jobs: + lint: + name: Lint, Format & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install dev dependencies + run: pip install -e ".[dev]" + - name: ruff lint + run: ruff check . + - name: ruff format check + run: ruff format --check . + - name: mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy {mod}/ + fi + + test: + name: Test (Python ${{ matrix.python-version }}) + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10", "3.11", "3.12", "3.13"] + + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED for setuptools_scm to read git tags + + - uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install dependencies + run: pip install -e ".[dev]" + + - name: Run tests with coverage + run: pytest --cov --cov-report=xml + + - name: Upload coverage + uses: codecov/codecov-action@v4 + with: + token: ${{ secrets.CODECOV_TOKEN }} + fail_ci_if_error: false + + test-redis: + name: Test Redis backend + runs-on: ubuntu-latest + services: + redis: + image: redis:7-alpine + ports: ["6379:6379"] + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install with Redis extra + run: pip install -e ".[dev,redis]" + + - name: Run Redis tests + run: pytest tests/test_redis_backend.py -v +``` + +> **Always add `fetch-depth: 0`** to every checkout step when using `setuptools_scm`. +> Without full git history, `setuptools_scm` can't find tags and the build fails with a version +> detection error. + +--- + +## 3. `publish.yml` + +Triggered automatically when you push a tag matching `v*.*.*`. Uses Trusted Publishing (OIDC) — +no API tokens in repository secrets. + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v*.*.*" + +jobs: + build: + name: Build distribution + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # Critical for setuptools_scm + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build tools + run: pip install build twine + + - name: Build package + run: python -m build + + - name: Check distribution + run: twine check dist/* + + - uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish: + name: Publish to PyPI + needs: build + runs-on: ubuntu-latest + environment: pypi + permissions: + id-token: write # Required for Trusted Publishing (OIDC) + + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +``` + +--- + +## 4. PyPI Trusted Publishing + +Trusted Publishing uses OpenID Connect (OIDC) so PyPI can verify that a publish came from your +specific GitHub Actions workflow — no long-lived API tokens required, no rotation burden. + +### One-time setup + +1. Create an account at https://pypi.org +2. Go to **Account → Publishing → Add a new pending publisher** +3. Fill in: + - GitHub owner (your username or org) + - Repository name + - Workflow filename: `publish.yml` + - Environment name: `pypi` +4. Create the `pypi` environment in GitHub: + **repo → Settings → Environments → New environment → name it `pypi`** + +That's it. The next time you push a `v*.*.*` tag, the workflow authenticates automatically. + +--- + +## 5. Manual Publish Fallback + +If CI isn't set up yet or you need to publish from your machine: + +```bash +pip install build twine + +# Build wheel + sdist +python -m build + +# Validate before uploading +twine check dist/* + +# Upload to PyPI +twine upload dist/* + +# OR test on TestPyPI first (recommended for first release) +twine upload --repository testpypi dist/* +pip install --index-url https://test.pypi.org/simple/ your-package +python -c "import your_package; print(your_package.__version__)" +``` + +--- + +## 6. Release Checklist + +``` +[ ] All tests pass on main/master +[ ] CHANGELOG.md updated — move [Unreleased] items to new version section with date +[ ] Update diff comparison links at bottom of CHANGELOG +[ ] git tag vX.Y.Z +[ ] git push origin master --tags +[ ] Monitor GitHub Actions publish.yml run +[ ] Verify on PyPI: pip install your-package==X.Y.Z +[ ] Test the installed version: + python -c "import your_package; print(your_package.__version__)" +``` + +--- + +## 7. Verify py.typed Ships in the Wheel + +After every build, confirm the typed marker is included: + +```bash +python -m build +unzip -l dist/your_package-*.whl | grep py.typed +# Must print: your_package/py.typed +# If missing, check [tool.setuptools.package-data] in pyproject.toml +``` + +If it's missing from the wheel, users won't get type information even though your code is +fully typed. This is a silent failure — always verify before releasing. + +--- + +## 8. Semver Change-Type Guide + +| Change | Version bump | Example | +|---|---|---| +| Breaking API change (remove/rename public symbol) | MAJOR | `1.2.3 → 2.0.0` | +| New feature, fully backward-compatible | MINOR | `1.2.3 → 1.3.0` | +| Bug fix, no API change | PATCH | `1.2.3 → 1.2.4` | +| Pre-release | suffix | `2.0.0a1 → 2.0.0rc1 → 2.0.0` | +| Packaging-only fix (no code change) | post-release | `1.2.3 → 1.2.3.post1` | diff --git a/skills/python-pypi-package-builder/references/community-docs.md b/skills/python-pypi-package-builder/references/community-docs.md new file mode 100644 index 000000000..ab970e1cf --- /dev/null +++ b/skills/python-pypi-package-builder/references/community-docs.md @@ -0,0 +1,411 @@ +# Community Docs, PR Checklist, Anti-patterns, and Release Checklist + +## Table of Contents +1. [README.md required sections](#1-readmemd-required-sections) +2. [Docstrings — Google style](#2-docstrings--google-style) +3. [CONTRIBUTING.md template](#3-contributingmd) +4. [SECURITY.md template](#4-securitymd) +5. [GitHub Issue Templates](#5-github-issue-templates) +6. [PR Checklist](#6-pr-checklist) +7. [Anti-patterns to avoid](#7-anti-patterns-to-avoid) +8. [Master Release Checklist](#8-master-release-checklist) + +--- + +## 1. `README.md` Required Sections + +A good README is the single most important file for adoption. Users decide in 30 seconds whether +to use your library based on the README. + +```markdown +# your-package + +> One-line description — what it does and why it's useful. + +[![PyPI version](https://badge.fury.io/py/your-package.svg)](https://pypi.org/project/your-package/) +[![Python Versions](https://img.shields.io/pypi/pyversions/your-package)](https://pypi.org/project/your-package/) +[![CI](https://github.com/you/your-package/actions/workflows/ci.yml/badge.svg)](https://github.com/you/your-package/actions/workflows/ci.yml) +[![Coverage](https://codecov.io/gh/you/your-package/branch/master/graph/badge.svg)](https://codecov.io/gh/you/your-package) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) + +## Installation + +pip install your-package + +# With Redis backend: +pip install "your-package[redis]" + +## Quick Start + +(A copy-paste working example — no setup required to run it) + +from your_package import YourClient + +client = YourClient(api_key="sk-...") +result = client.process({"input": "value"}) +print(result) + +## Features + +- Feature 1 +- Feature 2 + +## Configuration + +| Parameter | Type | Default | Description | +|---|---|---|—--| +| api_key | str | required | Authentication credential | +| timeout | int | 30 | Request timeout in seconds | +| retries | int | 3 | Number of retry attempts | + +## Backends + +Brief comparison — in-memory vs Redis — and when to use each. + +## Contributing + +See [CONTRIBUTING.md](./CONTRIBUTING.md) + +## Changelog + +See [CHANGELOG.md](./CHANGELOG.md) + +## License + +MIT — see [LICENSE](./LICENSE) +``` + +--- + +## 2. Docstrings — Google Style + +Use Google-style docstrings for every public class, method, and function. IDEs display these +as tooltips, mkdocs/sphinx can auto-generate documentation from them, and they convey intent +clearly to contributors. + +```python +class YourClient: + """ + Main client for . + + Args: + api_key: Authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from your_package import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process({"input": "value"}) + """ +``` + +--- + +## 3. `CONTRIBUTING.md` + +```markdown +# Contributing to your-package + +## Development Setup + +git clone https://github.com/you/your-package +cd your-package +pip install -e ".[dev]" +pre-commit install + +## Running Tests + +pytest + +## Running Linting + +ruff check . +black . --check +mypy your_package/ + +## Submitting a PR + +1. Fork the repository +2. Create a feature branch: `git checkout -b feat/your-feature` +3. Make changes with tests +4. Ensure CI passes: `pre-commit run --all-files && pytest` +5. Update `CHANGELOG.md` under `[Unreleased]` +6. Open a PR — use the PR template + +## Commit Message Format (Conventional Commits) + +- `feat: add Redis backend` +- `fix: correct retry behavior on timeout` +- `docs: update README quick start` +- `chore: bump ruff to 0.5` +- `test: add edge cases for memory backend` + +## Reporting Bugs + +Use the GitHub issue template. Include Python version, package version, +and a minimal reproducible example. +``` + +--- + +## 4. `SECURITY.md` + +```markdown +# Security Policy + +## Supported Versions + +| Version | Supported | +|---|---| +| 1.x.x | Yes | +| < 1.0 | No | + +## Reporting a Vulnerability + +Do NOT open a public GitHub issue for security vulnerabilities. + +Report via: GitHub private security reporting (preferred) +or email: security@yourdomain.com + +Include: +- Description of the vulnerability +- Steps to reproduce +- Potential impact +- Suggested fix (if any) + +We aim to acknowledge within 48 hours and resolve within 14 days. +``` + +--- + +## 5. GitHub Issue Templates + +### `.github/ISSUE_TEMPLATE/bug_report.md` + +```markdown +--- +name: Bug Report +about: Report a reproducible bug +labels: bug +--- + +**Python version:** +**Package version:** + +**Describe the bug:** + +**Minimal reproducible example:** +```python +# paste code here +``` + +**Expected behavior:** + +**Actual behavior:** +``` + +### `.github/ISSUE_TEMPLATE/feature_request.md` + +```markdown +--- +name: Feature Request +about: Suggest a new feature or enhancement +labels: enhancement +--- + +**Problem this would solve:** + +**Proposed solution:** + +**Alternatives considered:** +``` + +--- + +## 6. PR Checklist + +All items must be checked before requesting review. CI must be fully green. + +### Code Quality Gates +``` +[ ] ruff check . — zero errors +[ ] black . --check — zero formatting issues +[ ] isort . --check-only — imports sorted correctly +[ ] mypy your_package/ — zero type errors +[ ] pytest — all tests pass +[ ] Coverage >= 80% (enforced by fail_under in pyproject.toml) +[ ] All GitHub Actions workflows green +``` + +### Structure +``` +[ ] pyproject.toml: name, dynamic/version, description, requires-python, license, authors, + keywords (10+), classifiers, dependencies, all [project.urls] filled in +[ ] dynamic = ["version"] if using setuptools_scm +[ ] [tool.setuptools_scm] with local_scheme = "no-local-version" +[ ] setup.py shim present (if using setuptools_scm) +[ ] py.typed marker file exists in the package directory (empty file) +[ ] py.typed listed in [tool.setuptools.package-data] +[ ] "Typing :: Typed" classifier in pyproject.toml +[ ] __init__.py has __all__ listing all public symbols +[ ] __version__ via importlib.metadata (not hardcoded string) +``` + +### Testing +``` +[ ] conftest.py has shared fixtures for client and backend +[ ] Core happy path tested +[ ] Error conditions and edge cases tested +[ ] Each backend tested independently in isolation +[ ] Redis backend tested in separate CI job with redis service (if applicable) +[ ] asyncio_mode = "auto" in pyproject.toml (for async tests) +[ ] fetch-depth: 0 in all CI checkout steps +``` + +### Optional Backend (if applicable) +``` +[ ] BaseBackend abstract class defines the interface +[ ] MemoryBackend works with zero extra deps +[ ] RedisBackend raises ImportError with clear pip install hint if redis not installed +[ ] Both backends unit-tested independently +[ ] redis extra declared in [project.optional-dependencies] +[ ] README shows both install paths (base and [redis]) +``` + +### Changelog & Docs +``` +[ ] CHANGELOG.md updated under [Unreleased] +[ ] README has: description, install, quick start, config table, badges, license +[ ] All public symbols have Google-style docstrings +[ ] CONTRIBUTING.md: dev setup, test/lint commands, PR instructions +[ ] SECURITY.md: supported versions, reporting process +[ ] .github/ISSUE_TEMPLATE/bug_report.md +[ ] .github/ISSUE_TEMPLATE/feature_request.md +``` + +### CI/CD +``` +[ ] ci.yml: lint + mypy + test matrix (all supported Python versions) +[ ] ci.yml: separate job for Redis backend with redis service +[ ] publish.yml: triggered on v*.*.* tags, uses Trusted Publishing (OIDC) +[ ] fetch-depth: 0 in all workflow checkout steps +[ ] pypi environment created in GitHub repo Settings → Environments +[ ] No API tokens in repository secrets +``` + +--- + +## 7. Anti-patterns to Avoid + +| Anti-pattern | Why it's bad | Correct approach | +|---|---|---| +| `__version__ = "1.0.0"` hardcoded with setuptools_scm | Goes stale after first git tag | Use `importlib.metadata.version()` | +| Missing `fetch-depth: 0` in CI checkout | setuptools_scm can't find tags → version = `0.0.0+dev` | Add `fetch-depth: 0` to **every** checkout step | +| `local_scheme` not set | `+g` suffix breaks PyPI uploads (local versions rejected) | `local_scheme = "no-local-version"` | +| Missing `py.typed` file | IDEs and mypy don't see package as typed | Create empty `py.typed` in package root | +| `py.typed` not in `package-data` | File missing from installed wheel — useless | Add to `[tool.setuptools.package-data]` | +| Importing optional dep at module top | `ImportError` on `import your_package` for all users | Lazy import inside the function/class that needs it | +| Duplicating metadata in `setup.py` | Conflicts with `pyproject.toml`; drifts | Keep `setup.py` as 3-line shim only | +| No `fail_under` in coverage config | Coverage regressions go unnoticed | Set `fail_under = 80` | +| No mypy in CI | Type errors silently accumulate | Add mypy step to `ci.yml` | +| API tokens in GitHub Secrets for PyPI | Security risk, rotation burden | Use Trusted Publishing (OIDC) | +| Committing directly to `main`/`master` | Bypasses CI checks | Enforce via `no-commit-to-branch` pre-commit hook | +| Missing `[Unreleased]` section in CHANGELOG | Changes pile up and get forgotten at release time | Keep `[Unreleased]` updated every PR | +| Pinning exact dep versions in a library | Breaks dependency resolution for users | Use `>=` lower bounds only; avoid `==` | +| No `__all__` in `__init__.py` | Users can accidentally import internal helpers | Declare `__all__` with every public symbol | +| `from your_package import *` in tests | Tests pass even when imports are broken | Always use explicit imports | +| No `SECURITY.md` | No path for responsible vulnerability disclosure | Add file with response timeline | +| `Any` everywhere in type hints | Defeats mypy entirely | Use `object` for truly arbitrary values | +| `Union` return types | Forces every caller to write `isinstance()` checks | Return concrete types; use overloads | +| `setup.cfg` + `pyproject.toml` both active | Conflicts and confusing for contributors | Migrate everything to `pyproject.toml` | +| Releasing on untagged commits | Version number is meaningless | Always tag before release | +| Not testing on all supported Python versions | Breakage discovered by users, not you | Matrix test in CI | +| `license = {text = "MIT"}` (old form) | Deprecated; PEP 639 uses SPDX strings | `license = "MIT"` | +| No issue templates | Bug reports are inconsistent | Add `bug_report.md` + `feature_request.md` | + +--- + +## 8. Master Release Checklist + +Run through every item before pushing a release tag. CI must be fully green. + +### Code Quality +``` +[ ] ruff check . — zero errors +[ ] ruff format . --check — zero formatting issues +[ ] mypy src/your_package/ — zero type errors +[ ] pytest — all tests pass +[ ] Coverage >= 80% (fail_under enforced in pyproject.toml) +[ ] All GitHub Actions CI jobs green (lint + test matrix) +``` + +### Project Structure +``` +[ ] pyproject.toml — name, description, requires-python, license (SPDX string), authors, + keywords (10+), classifiers (Python versions + Typing :: Typed), urls (all 5 fields) +[ ] dynamic = ["version"] set (if using setuptools_scm or hatch-vcs) +[ ] [tool.setuptools_scm] with local_scheme = "no-local-version" +[ ] setup.py shim present (if using setuptools_scm) +[ ] py.typed marker file exists (empty file in package root) +[ ] py.typed listed in [tool.setuptools.package-data] +[ ] "Typing :: Typed" classifier in pyproject.toml +[ ] __init__.py has __all__ listing all public symbols +[ ] __version__ reads from importlib.metadata (not hardcoded) +``` + +### Testing +``` +[ ] conftest.py has shared fixtures for client and backend +[ ] Core happy path tested +[ ] Error conditions and edge cases tested +[ ] Each backend tested independently in isolation +[ ] asyncio_mode = "auto" in pyproject.toml (for async tests) +[ ] fetch-depth: 0 in all CI checkout steps +``` + +### CHANGELOG and Docs +``` +[ ] CHANGELOG.md: [Unreleased] entries moved to [x.y.z] - YYYY-MM-DD +[ ] README has: description, install commands, quick start, config table, badges +[ ] All public symbols have Google-style docstrings +[ ] CONTRIBUTING.md: dev setup, test/lint commands, PR instructions +[ ] SECURITY.md: supported versions, reporting process with timeline +``` + +### Versioning +``` +[ ] All CI checks pass on the commit you plan to tag +[ ] CHANGELOG.md updated and committed +[ ] Git tag follows format v1.2.3 (semver, v prefix) +[ ] No stale local_scheme suffixes will appear in the built wheel name +``` + +### CI/CD +``` +[ ] ci.yml: lint + mypy + test matrix (all supported Python versions) +[ ] publish.yml: triggered on v*.*.* tags, uses Trusted Publishing (OIDC) +[ ] pypi environment created in GitHub repo Settings → Environments +[ ] No API tokens stored in repository secrets +``` + +### The Release Command Sequence +```bash +# 1. Run full local validation +ruff check . ; ruff format . --check ; mypy src/your_package/ ; pytest + +# 2. Update CHANGELOG.md — move [Unreleased] to [x.y.z] +# 3. Commit the changelog +git add CHANGELOG.md +git commit -m "chore: prepare release vX.Y.Z" + +# 4. Tag and push — this triggers publish.yml automatically +git tag vX.Y.Z +git push origin main --tags + +# 5. Monitor: https://github.com///actions +# 6. Verify: https://pypi.org/project/your-package/ +``` diff --git a/skills/python-pypi-package-builder/references/library-patterns.md b/skills/python-pypi-package-builder/references/library-patterns.md new file mode 100644 index 000000000..d1ec780ce --- /dev/null +++ b/skills/python-pypi-package-builder/references/library-patterns.md @@ -0,0 +1,606 @@ +# Library Core Patterns, OOP/SOLID, and Type Hints + +## Table of Contents +1. [OOP & SOLID Principles](#1-oop--solid-principles) +2. [Type Hints Best Practices](#2-type-hints-best-practices) +3. [Core Class Design](#3-core-class-design) +4. [Factory / Builder Pattern](#4-factory--builder-pattern) +5. [Configuration Pattern](#5-configuration-pattern) +6. [`__init__.py` — explicit public API](#6-__init__py--explicit-public-api) +7. [Optional Backends (Plugin Pattern)](#7-optional-backends-plugin-pattern) + +--- + +## 1. OOP & SOLID Principles + +Apply these principles to produce maintainable, testable, extensible packages. +**Do not over-engineer** — apply the principle that solves a real problem, not all of them +at once. + +### S — Single Responsibility Principle + +Each class/module should have **one reason to change**. + +```python +# BAD: one class handles data, validation, AND persistence +class UserManager: + def validate(self, user): ... + def save_to_db(self, user): ... + def send_email(self, user): ... + +# GOOD: split responsibilities +class UserValidator: + def validate(self, user: User) -> None: ... + +class UserRepository: + def save(self, user: User) -> None: ... + +class UserNotifier: + def notify(self, user: User) -> None: ... +``` + +### O — Open/Closed Principle + +Open for extension, closed for modification. Use **protocols or ABCs** as extension points. + +```python +from abc import ABC, abstractmethod + +class StorageBackend(ABC): + """Define the interface once; never modify it for new implementations.""" + @abstractmethod + def get(self, key: str) -> str | None: ... + @abstractmethod + def set(self, key: str, value: str) -> None: ... + +class MemoryBackend(StorageBackend): # Extend by subclassing + ... + +class RedisBackend(StorageBackend): # Add new impl without touching StorageBackend + ... +``` + +### L — Liskov Substitution Principle + +Subclasses must be substitutable for their base. Never narrow a contract in a subclass. + +```python +class BaseProcessor: + def process(self, data: dict) -> dict: ... + +# BAD: raises TypeError for valid dicts — breaks substitutability +class StrictProcessor(BaseProcessor): + def process(self, data: dict) -> dict: + if not data: + raise TypeError("Must have data") # Base never raised this + +# GOOD: accept what base accepts, fulfill the same contract +class StrictProcessor(BaseProcessor): + def process(self, data: dict) -> dict: + if not data: + return {} # Graceful — same return type, no new exceptions +``` + +### I — Interface Segregation Principle + +Prefer **small, focused protocols** over large monolithic ABCs. + +```python +# BAD: forces all implementers to handle read+write+delete+list +class BigStorage(ABC): + @abstractmethod + def read(self): ... + @abstractmethod + def write(self): ... + @abstractmethod + def delete(self): ... + @abstractmethod + def list_all(self): ... # Not every backend needs this + +# GOOD: separate protocols — clients depend only on what they need +from typing import Protocol + +class Readable(Protocol): + def read(self, key: str) -> str | None: ... + +class Writable(Protocol): + def write(self, key: str, value: str) -> None: ... + +class Deletable(Protocol): + def delete(self, key: str) -> None: ... +``` + +### D — Dependency Inversion Principle + +High-level modules depend on **abstractions** (protocols/ABCs), not concrete implementations. +Pass dependencies in via `__init__` (constructor injection). + +```python +# BAD: high-level class creates its own dependency +class ApiClient: + def __init__(self) -> None: + self._cache = RedisCache() # Tightly coupled to Redis + +# GOOD: depend on the abstraction; inject the concrete at call site +class ApiClient: + def __init__(self, cache: CacheBackend) -> None: # CacheBackend is a Protocol + self._cache = cache + +# User code (or tests): +client = ApiClient(cache=RedisCache()) # Real +client = ApiClient(cache=MemoryCache()) # Test +``` + +### Composition Over Inheritance + +Prefer delegating to contained objects over deep inheritance chains. + +```python +# Prefer this (composition): +class YourClient: + def __init__(self, backend: StorageBackend, http: HttpTransport) -> None: + self._backend = backend + self._http = http + +# Avoid this (deep inheritance): +class YourClient(BaseClient, CacheMixin, RetryMixin, LoggingMixin): + ... # Fragile, hard to test, MRO confusion +``` + +### Exception Hierarchy + +Always define a base exception for your package; layer specifics below it. + +```python +# your_package/exceptions.py +class YourPackageError(Exception): + """Base exception — catch this to catch any package error.""" + +class ConfigurationError(YourPackageError): + """Raised when package is misconfigured.""" + +class AuthenticationError(YourPackageError): + """Raised on auth failure.""" + +class RateLimitError(YourPackageError): + """Raised when rate limit is exceeded.""" + def __init__(self, retry_after: int) -> None: + self.retry_after = retry_after + super().__init__(f"Rate limited. Retry after {retry_after}s.") +``` + +--- + +## 2. Type Hints Best Practices + +Follow PEP 484 (type hints), PEP 526 (variable annotations), PEP 544 (protocols), +PEP 561 (typed packages). These are not optional for a quality library. + +```python +from __future__ import annotations # Enables PEP 563 deferred evaluation — always add this + +# For ARGUMENTS: prefer abstract / protocol types (more flexible for callers) +from collections.abc import Iterable, Mapping, Sequence, Callable + +def process_items(items: Iterable[str]) -> list[int]: ... # ✓ Accepts any iterable +def process_items(items: list[str]) -> list[int]: ... # ✗ Too restrictive + +# For RETURN TYPES: prefer concrete types (callers know exactly what they get) +def get_names() -> list[str]: ... # ✓ Concrete +def get_names() -> Iterable[str]: ... # ✗ Caller can't index it + +# Use X | Y syntax (Python 3.10+), not Union[X, Y] or Optional[X] +def find(key: str) -> str | None: ... # ✓ Modern +def find(key: str) -> Optional[str]: ... # ✗ Old style + +# None should be LAST in unions +def get(key: str) -> str | int | None: ... # ✓ + +# Avoid Any — it disables type checking entirely +def process(data: Any) -> Any: ... # ✗ Loses all safety +def process(data: dict[str, object]) -> dict[str, object]: # ✓ + +# Use object instead of Any when a param accepts literally anything +def log(value: object) -> None: ... # ✓ + +# Avoid Union return types — they require isinstance() checks at every call site +def get_value() -> str | int: ... # ✗ Forces callers to branch +``` + +### Protocols vs ABCs + +```python +from typing import Protocol, runtime_checkable +from abc import ABC, abstractmethod + +# Use Protocol when you don't control the implementer classes (duck typing) +@runtime_checkable # Makes isinstance() checks work at runtime +class Serializable(Protocol): + def to_dict(self) -> dict[str, object]: ... + +# Use ABC when you control the class hierarchy and want default implementations +class BaseBackend(ABC): + @abstractmethod + async def get(self, key: str) -> str | None: ... + + def get_or_default(self, key: str, default: str) -> str: + result = self.get(key) + return result if result is not None else default +``` + +### TypeVar and Generics + +```python +from typing import TypeVar, Generic + +T = TypeVar("T") +T_co = TypeVar("T_co", covariant=True) # For read-only containers + +class Repository(Generic[T]): + """Type-safe generic repository.""" + def __init__(self, model_class: type[T]) -> None: + self._store: list[T] = [] + + def add(self, item: T) -> None: + self._store.append(item) + + def get_all(self) -> list[T]: + return list(self._store) +``` + +### dataclasses for data containers + +```python +from dataclasses import dataclass, field + +@dataclass(frozen=True) # frozen=True → immutable, hashable (good for configs/keys) +class Config: + api_key: str + timeout: int = 30 + headers: dict[str, str] = field(default_factory=dict) + + def __post_init__(self) -> None: + if not self.api_key: + raise ValueError("api_key must not be empty") +``` + +### TYPE_CHECKING guard (avoid circular imports) + +```python +from __future__ import annotations +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from your_package.models import HeavyModel # Only imported during type checking + +def process(model: "HeavyModel") -> None: + ... +``` + +### Overload for multiple signatures + +```python +from typing import overload + +@overload +def get(key: str, default: None = ...) -> str | None: ... +@overload +def get(key: str, default: str) -> str: ... +def get(key: str, default: str | None = None) -> str | None: + ... # Single implementation handles both +``` + +--- + +## 3. Core Class Design + +The main class of your library should have a clear, minimal `__init__`, sensible defaults for all +parameters, and raise `TypeError` / `ValueError` early for invalid inputs. This prevents confusing +errors at call time rather than at construction. + +```python +# your_package/core.py +from __future__ import annotations + +from your_package.exceptions import YourPackageError + + +class YourClient: + """ + Main entry point for . + + Args: + api_key: Required authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from your_package import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process(data) + """ + + def __init__( + self, + api_key: str, + timeout: int = 30, + retries: int = 3, + ) -> None: + if not api_key: + raise ValueError("api_key must not be empty") + if timeout <= 0: + raise ValueError("timeout must be positive") + self._api_key = api_key + self.timeout = timeout + self.retries = retries + + def process(self, data: dict) -> dict: + """ + Process data and return results. + + Args: + data: Input dictionary to process. + + Returns: + Processed result as a dictionary. + + Raises: + YourPackageError: If processing fails. + """ + ... +``` + +### Design rules + +- Accept all config in `__init__`, not scattered across method calls. +- Validate at construction time — fail fast with a clear message. +- Keep `__init__` signatures stable. Adding new **keyword-only** args with defaults is backwards + compatible. Removing or reordering positional args is a breaking change. + +--- + +## 4. Factory / Builder Pattern + +Use a factory function when users need to create pre-configured instances. This avoids cluttering +`__init__` with a dozen keyword arguments and keeps the common case simple. + +```python +# your_package/factory.py +from __future__ import annotations + +from your_package.core import YourClient +from your_package.backends.memory import MemoryBackend + + +def create_client( + api_key: str, + *, + timeout: int = 30, + retries: int = 3, + backend: str = "memory", + backend_url: str | None = None, +) -> YourClient: + """ + Factory that returns a configured YourClient. + + Args: + api_key: Required API key. + timeout: Request timeout in seconds. + retries: Number of retry attempts. + backend: Storage backend type. One of 'memory' or 'redis'. + backend_url: Connection URL for the chosen backend. + + Example: + >>> client = create_client(api_key="sk-...", backend="redis", backend_url="redis://localhost") + """ + if backend == "redis": + from your_package.backends.redis import RedisBackend + _backend = RedisBackend(url=backend_url or "redis://localhost:6379") + else: + _backend = MemoryBackend() + + return YourClient(api_key=api_key, timeout=timeout, retries=retries, backend=_backend) +``` + +**Why a factory, not a class method?** Both work. A standalone factory function is easier to +mock in tests and avoids coupling the factory logic into the class itself. + +--- + +## 5. Configuration Pattern + +Use a dataclass (or Pydantic `BaseModel`) to hold configuration. This gives you free validation, +helpful error messages, and a single place to document every option. + +```python +# your_package/config.py +from __future__ import annotations +from dataclasses import dataclass, field + + +@dataclass +class YourSettings: + """ + Configuration for YourClient. + + Attributes: + timeout: HTTP timeout in seconds. + retries: Number of retry attempts on transient errors. + base_url: Base API URL. + """ + timeout: int = 30 + retries: int = 3 + base_url: str = "https://api.example.com" + extra_headers: dict[str, str] = field(default_factory=dict) + + def __post_init__(self) -> None: + if self.timeout <= 0: + raise ValueError("timeout must be positive") + if self.retries < 0: + raise ValueError("retries must be non-negative") +``` + +If you need environment variable loading, use `pydantic-settings` as an **optional** dependency — +declare it in `[project.optional-dependencies]`, not as a required dep. + +--- + +## 6. `__init__.py` — Explicit Public API + +A well-defined `__all__` is not just style — it tells users (and IDEs) exactly what's part of your +public API, and prevents accidental imports of internal helpers as part of your contract. + +```python +# your_package/__init__.py +"""your-package: .""" + +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" + +from your_package.core import YourClient +from your_package.config import YourSettings +from your_package.exceptions import YourPackageError + +__all__ = [ + "YourClient", + "YourSettings", + "YourPackageError", + "__version__", +] +``` + +Rules: +- Only export what users are supposed to use. Internal helpers go in `_utils.py` or submodules. +- Keep imports at the top level of `__init__.py` shallow — avoid importing heavy optional deps + (like `redis`) at module level. Import them lazily inside the class or function that needs them. +- `__version__` is always part of the public API — it enables `your_package.__version__` for + debugging. + +--- + +## 7. Optional Backends (Plugin Pattern) + +This pattern lets your package work out-of-the-box (no extra deps) with an in-memory backend, +while letting advanced users plug in Redis, a database, or any custom storage. + +### 5.1 Abstract base class — defines the interface + +```python +# your_package/backends/__init__.py +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage backend interface. + + Implement this to add a custom backend (database, cache, etc.). + """ + + @abstractmethod + async def get(self, key: str) -> str | None: + """Retrieve a value by key. Returns None if not found.""" + ... + + @abstractmethod + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value. Optional TTL in seconds.""" + ... + + @abstractmethod + async def delete(self, key: str) -> None: + """Delete a key.""" + ... +``` + +### 5.2 Memory backend — zero extra deps + +```python +# your_package/backends/memory.py +from __future__ import annotations + +import asyncio +import time +from your_package.backends import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. Works out of the box — no extra dependencies.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = asyncio.Lock() + + async def get(self, key: str) -> str | None: + async with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.time() > expires_at: + del self._store[key] + return None + return value + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + async with self._lock: + expires_at = time.time() + ttl if ttl is not None else None + self._store[key] = (value, expires_at) + + async def delete(self, key: str) -> None: + async with self._lock: + self._store.pop(key, None) +``` + +### 5.3 Redis backend — raises clear ImportError if not installed + +The key design: import `redis` lazily inside `__init__`, not at module level. This way, +`import your_package` never fails even if `redis` isn't installed. + +```python +# your_package/backends/redis.py +from __future__ import annotations +from your_package.backends import BaseBackend + +try: + import redis.asyncio as aioredis +except ImportError as exc: + raise ImportError( + "Redis backend requires the redis extra:\n" + " pip install your-package[redis]" + ) from exc + + +class RedisBackend(BaseBackend): + """Redis-backed storage for distributed/multi-process deployments.""" + + def __init__(self, url: str = "redis://localhost:6379") -> None: + self._client = aioredis.from_url(url, decode_responses=True) + + async def get(self, key: str) -> str | None: + return await self._client.get(key) + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + await self._client.set(key, value, ex=ttl) + + async def delete(self, key: str) -> None: + await self._client.delete(key) +``` + +### 5.4 How users choose a backend + +```python +# Default: in-memory, no extra deps needed +from your_package import YourClient +client = YourClient(api_key="sk-...") + +# Redis: pip install your-package[redis] +from your_package.backends.redis import RedisBackend +client = YourClient(api_key="sk-...", backend=RedisBackend(url="redis://localhost:6379")) +``` diff --git a/skills/python-pypi-package-builder/references/pyproject-toml.md b/skills/python-pypi-package-builder/references/pyproject-toml.md new file mode 100644 index 000000000..eccfac967 --- /dev/null +++ b/skills/python-pypi-package-builder/references/pyproject-toml.md @@ -0,0 +1,470 @@ +# pyproject.toml, Backends, Versioning, and Typed Package + +## Table of Contents +1. [Complete pyproject.toml — setuptools + setuptools_scm](#1-complete-pyprojecttoml) +2. [hatchling (modern, zero-config)](#2-hatchling-modern-zero-config) +3. [flit (minimal, version from `__version__`)](#3-flit-minimal-version-from-__version__) +4. [poetry (integrated dep manager)](#4-poetry-integrated-dep-manager) +5. [Versioning Strategy — PEP 440, semver, dep specifiers](#5-versioning-strategy) +6. [setuptools_scm — dynamic version from git tags](#6-dynamic-versioning-with-setuptools_scm) +7. [setup.py shim for legacy editable installs](#7-setuppy-shim) +8. [PEP 561 typed package (py.typed)](#8-typed-package-pep-561) + +--- + +## 1. Complete pyproject.toml + +### setuptools + setuptools_scm (recommended for git-tag versioning) + +```toml +[build-system] +requires = ["setuptools>=68", "wheel", "setuptools_scm"] +build-backend = "setuptools.build_meta" + +[project] +name = "your-package" +dynamic = ["version"] # Version comes from git tags via setuptools_scm +description = ", " +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" # PEP 639 SPDX expression (string, not {text = "MIT"}) +license-files = ["LICENSE"] +authors = [ + {name = "Your Name", email = "you@example.com"}, +] +maintainers = [ + {name = "Your Name", email = "you@example.com"}, +] +keywords = [ + "python", + # Add 10-15 specific keywords that describe your library — they affect PyPI discoverability +] +classifiers = [ + "Development Status :: 3 - Alpha", # Change to 5 at stable release + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Topic :: Software Development :: Libraries :: Python Modules", + "Typing :: Typed", # Add this when shipping py.typed +] +dependencies = [ + # List your runtime dependencies here. Keep them minimal. + # Example: "httpx>=0.24", "pydantic>=2.0" + # Leave empty if your library has no required runtime deps. +] + +[project.optional-dependencies] +redis = [ + "redis>=4.2", # Optional heavy backend +] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" +Documentation = "https://github.com/yourusername/your-package#readme" +Repository = "https://github.com/yourusername/your-package" +"Bug Tracker" = "https://github.com/yourusername/your-package/issues" +Changelog = "https://github.com/yourusername/your-package/blob/master/CHANGELOG.md" + +# --- Setuptools configuration --- +[tool.setuptools.packages.find] +include = ["your_package*"] # flat layout +# For src/ layout, use: +# where = ["src"] + +[tool.setuptools.package-data] +your_package = ["py.typed"] # Ship the py.typed marker in the wheel + +# --- setuptools_scm: version from git tags --- +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +local suffix breaking PyPI uploads + +# --- Ruff (linting) --- +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "N", "UP", "B", "SIM", "C4", "PTH", "RUF"] +ignore = ["E501"] # Line length enforced by formatter + +[tool.ruff.lint.per-file-ignores] +"tests/*" = ["S101", "ANN"] # Allow assert and missing annotations in tests +"scripts/*" = ["T201"] # Allow print in scripts + +[tool.ruff.format] +quote-style = "double" + +# --- Black (formatting) --- +[tool.black] +line-length = 100 +target-version = ["py310", "py311", "py312", "py313"] + +# --- isort (import sorting) --- +[tool.isort] +profile = "black" +line_length = 100 + +# --- mypy (static type checking) --- +[tool.mypy] +python_version = "3.10" +warn_return_any = true +warn_unused_configs = true +warn_unused_ignores = true +disallow_untyped_defs = true +disallow_any_generics = true +ignore_missing_imports = true +strict = false # Set true for maximum strictness + +[[tool.mypy.overrides]] +module = "tests.*" +disallow_untyped_defs = false # Relaxed in tests + +# --- pytest --- +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"] +pythonpath = ["."] # For flat layout; remove for src/ +python_files = "test_*.py" +python_classes = "Test*" +python_functions = "test_*" +addopts = "-v --tb=short --cov=your_package --cov-report=term-missing" + +# --- Coverage --- +[tool.coverage.run] +source = ["your_package"] +omit = ["tests/*"] + +[tool.coverage.report] +fail_under = 80 +show_missing = true +exclude_lines = [ + "pragma: no cover", + "def __repr__", + "raise NotImplementedError", + "if TYPE_CHECKING:", + "@abstractmethod", +] +``` + +--- + +## 2. hatchling (Modern, Zero-Config) + +Best for new pure-Python projects that don't need C extensions. No `setup.py` needed. Use +`hatch-vcs` for git-tag versioning, or omit it for manual version bumps. + +```toml +[build-system] +requires = ["hatchling", "hatch-vcs"] # hatch-vcs for git-tag versioning +build-backend = "hatchling.build" + +[project] +name = "your-package" +dynamic = ["version"] # Remove and add version = "1.0.0" for manual versioning +description = "One-line description" +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +license-files = ["LICENSE"] +authors = [{name = "Your Name", email = "you@example.com"}] +keywords = ["python"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] + +[project.optional-dependencies] +dev = ["pytest>=8.0", "pytest-cov>=5.0", "ruff>=0.6", "mypy>=1.10"] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" +Changelog = "https://github.com/yourusername/your-package/blob/master/CHANGELOG.md" + +# --- Hatchling build config --- +[tool.hatch.build.targets.wheel] +packages = ["src/your_package"] # src/ layout +# packages = ["your_package"] # ← flat layout + +[tool.hatch.version] +source = "vcs" # git-tag versioning via hatch-vcs + +[tool.hatch.version.raw-options] +local_scheme = "no-local-version" + +# ruff, mypy, pytest, coverage sections — same as setuptools template above +``` + +--- + +## 3. flit (Minimal, Version from `__version__`) + +Best for very simple, single-module packages. Zero config. Version is read directly from +`your_package/__init__.py`. Always requires a **static string** for `__version__`. + +```toml +[build-system] +requires = ["flit_core>=3.9"] +build-backend = "flit_core.buildapi" + +[project] +name = "your-package" +dynamic = ["version", "description"] # Read from __init__.py __version__ and docstring +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +authors = [{name = "Your Name", email = "you@example.com"}] +classifiers = [ + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" + +# flit reads __version__ from your_package/__init__.py automatically. +# Ensure __init__.py has: __version__ = "1.0.0" (static string — flit does NOT support +# importlib.metadata for dynamic version discovery) +``` + +--- + +## 4. poetry (Integrated Dependency + Build Manager) + +Best for teams that want a single tool to manage deps, build, and publish. Poetry v2+ +supports the standard `[project]` table. + +```toml +[build-system] +requires = ["poetry-core>=2.0"] +build-backend = "poetry.core.masonry.api" + +[project] +name = "your-package" +version = "1.0.0" +description = "One-line description" +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +authors = [{name = "Your Name", email = "you@example.com"}] +classifiers = [ + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] # poetry v2+ uses standard [project] table + +[project.optional-dependencies] +dev = ["pytest>=8.0", "ruff>=0.6", "mypy>=1.10"] + +# Optional: use [tool.poetry] only for poetry-specific features +[tool.poetry.group.dev.dependencies] +# Poetry-specific group syntax (alternative to [project.optional-dependencies]) +pytest = ">=8.0" +``` + +--- + +## 5. Versioning Strategy + +### PEP 440 — The Standard + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +Examples: + 1.0.0 Stable release + 1.0.0a1 Alpha (pre-release) + 1.0.0b2 Beta + 1.0.0rc1 Release candidate + 1.0.0.post1 Post-release (e.g., packaging fix only — no code change) + 1.0.0.dev1 Development snapshot (NOT for PyPI) +``` + +### Semantic Versioning (SemVer) — use this for every library + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking API change (remove/rename public function/class/arg) +MINOR: New feature, fully backward-compatible +PATCH: Bug fix, no API change +``` + +| Change | What bumps | Example | +|---|---|---| +| Remove / rename a public function | MAJOR | `1.2.3 → 2.0.0` | +| Add new public function | MINOR | `1.2.3 → 1.3.0` | +| Bug fix, no API change | PATCH | `1.2.3 → 1.2.4` | +| New pre-release | suffix | `2.0.0a1`, `2.0.0rc1` | + +### Version in code — read from package metadata + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Never hardcode `__version__ = "1.0.0"` when using setuptools_scm — it goes stale after the +first git tag. Use `importlib.metadata` always. + +### Version specifier best practices for dependencies + +```toml +# In [project] dependencies — for a LIBRARY: +"httpx>=0.24" # Minimum version — PREFERRED for libraries +"httpx>=0.24,<1.0" # Upper bound only when a known breaking change exists + +# ONLY for applications (never for libraries): +"httpx==0.27.0" # Pin exactly — breaks dep resolution in libraries + +# NEVER do this in a library: +# "httpx~=0.24.0" # Compatible release operator — too tight +# "httpx==0.27.*" # Wildcard pin — fragile +``` + +--- + +## 6. Dynamic Versioning with `setuptools_scm` + +`setuptools_scm` reads your git tags and sets the package version automatically — no more manually +editing version strings before each release. + +### How it works + +``` +git tag v1.0.0 → package version = 1.0.0 +git tag v1.1.0 → package version = 1.1.0 +(commits after tag) → version = 1.1.0.post1+g (stripped for PyPI) +``` + +`local_scheme = "no-local-version"` strips the `+g` suffix so PyPI uploads never fail with +a "local version label not allowed" error. + +### Access version at runtime + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Never hardcode `__version__ = "1.0.0"` when using setuptools_scm — it will go stale after the +first tag. + +### Full release flow (this is it — nothing else needed) + +```bash +git tag v1.2.0 +git push origin master --tags +# GitHub Actions publish.yml triggers automatically +``` + +--- + +## 7. `setup.py` Shim + +Some older tools and IDEs still expect a `setup.py`. Keep it as a three-line shim — all real +configuration stays in `pyproject.toml`. + +```python +# setup.py — thin shim only. All config lives in pyproject.toml. +from setuptools import setup + +setup() +``` + +Never duplicate `name`, `version`, `dependencies`, or any other metadata from `pyproject.toml` +into `setup.py`. If you copy anything there it will eventually drift and cause confusing conflicts. + +--- + +## 8. Typed Package (PEP 561) + +A properly declared typed package means mypy, pyright, and IDEs automatically pick up your type +hints without any extra configuration from your users. + +### Step 1: Create the marker file + +```bash +# The file must exist; its content doesn't matter — its presence is the signal. +touch your_package/py.typed +``` + +### Step 2: Include it in the wheel + +Already in the template above: + +```toml +[tool.setuptools.package-data] +your_package = ["py.typed"] +``` + +### Step 3: Add the PyPI classifier + +```toml +classifiers = [ + ... + "Typing :: Typed", +] +``` + +### Step 4: Type-annotate all public functions + +```python +# Good — fully typed +def process( + self, + data: dict[str, object], + *, + timeout: int = 30, +) -> dict[str, object]: + ... + +# Bad — mypy will flag this, and IDEs give no completions to users +def process(self, data, timeout=30): + ... +``` + +### Step 5: Verify py.typed ships in the wheel + +```bash +python -m build +unzip -l dist/your_package-*.whl | grep py.typed +# Must show: your_package/py.typed +``` + +If it's missing, check your `[tool.setuptools.package-data]` config. diff --git a/skills/python-pypi-package-builder/references/release-governance.md b/skills/python-pypi-package-builder/references/release-governance.md new file mode 100644 index 000000000..5df7c4e56 --- /dev/null +++ b/skills/python-pypi-package-builder/references/release-governance.md @@ -0,0 +1,354 @@ +# Release Governance — Branching, Protection, OIDC, and Access Control + +## Table of Contents +1. [Branch Strategy](#1-branch-strategy) +2. [Branch Protection Rules](#2-branch-protection-rules) +3. [Tag-Based Release Model](#3-tag-based-release-model) +4. [Role-Based Access Control](#4-role-based-access-control) +5. [Secure Publishing with OIDC (Trusted Publishing)](#5-secure-publishing-with-oidc-trusted-publishing) +6. [Validate Tag Author in CI](#6-validate-tag-author-in-ci) +7. [Prevent Invalid Release Tags](#7-prevent-invalid-release-tags) +8. [Full `publish.yml` with Governance Gates](#8-full-publishyml-with-governance-gates) + +--- + +## 1. Branch Strategy + +Use a clear branch hierarchy to separate development work from releasable code. + +``` +main ← stable; only receives PRs from develop or hotfix/* +develop ← integration branch; all feature PRs merge here first +feature/* ← new capabilities (e.g., feature/add-redis-backend) +fix/* ← bug fixes (e.g., fix/memory-leak-on-close) +hotfix/* ← urgent production fixes; PR directly to main + cherry-pick to develop +release/* ← (optional) release preparation (e.g., release/v2.0.0) +``` + +### Rules + +| Rule | Why | +|---|---| +| No direct push to `main` | Prevent accidental breakage of the stable branch | +| All changes via PR | Enforces review + CI before merge | +| At least one approval required | Second pair of eyes on all changes | +| CI must pass | Never merge broken code | +| Only tags trigger releases | No ad-hoc publish from branch pushes | + +--- + +## 2. Branch Protection Rules + +Configure these in **GitHub → Settings → Branches → Add rule** for `main` and `develop`. + +### For `main` + +```yaml +# Equivalent GitHub branch protection config (for documentation) +branch: main +rules: + - require_pull_request_reviews: + required_approving_review_count: 1 + dismiss_stale_reviews: true + - require_status_checks_to_pass: + contexts: + - "Lint, Format & Type Check" + - "Test (Python 3.11)" # at minimum; add all matrix versions + strict: true # branch must be up-to-date before merge + - restrict_pushes: + allowed_actors: [] # nobody — only PR merges + - require_linear_history: true # prevents merge commits on main +``` + +### For `develop` + +```yaml +branch: develop +rules: + - require_pull_request_reviews: + required_approving_review_count: 1 + - require_status_checks_to_pass: + contexts: ["CI"] + strict: false # less strict for the integration branch +``` + +### Via GitHub CLI + +```bash +# Protect main (requires gh CLI and admin rights) +gh api repos/{owner}/{repo}/branches/main/protection \ + --method PUT \ + --input - <<'EOF' +{ + "required_status_checks": { + "strict": true, + "contexts": ["Lint, Format & Type Check", "Test (Python 3.11)"] + }, + "enforce_admins": false, + "required_pull_request_reviews": { + "required_approving_review_count": 1, + "dismiss_stale_reviews": true + }, + "restrictions": null +} +EOF +``` + +--- + +## 3. Tag-Based Release Model + +**Only annotated tags on `main` trigger a release.** Branch pushes and PR merges never publish. + +### Tag Naming Convention + +``` +vMAJOR.MINOR.PATCH # Stable: v1.2.3 +vMAJOR.MINOR.PATCHaN # Alpha: v2.0.0a1 +vMAJOR.MINOR.PATCHbN # Beta: v2.0.0b1 +vMAJOR.MINOR.PATCHrcN # Release Candidate: v2.0.0rc1 +``` + +### Release Workflow + +```bash +# 1. Merge develop → main via PR (reviewed, CI green) + +# 2. Update CHANGELOG.md on main +# Move [Unreleased] entries to [vX.Y.Z] - YYYY-MM-DD + +# 3. Commit the changelog +git checkout main +git pull origin main +git add CHANGELOG.md +git commit -m "chore: release v1.2.3" + +# 4. Create and push an annotated tag +git tag -a v1.2.3 -m "Release v1.2.3" +git push origin v1.2.3 # ← ONLY the tag; not --tags (avoids pushing all tags) + +# 5. Confirm: GitHub Actions publish.yml triggers automatically +# Monitor: Actions tab → publish workflow +# Verify: https://pypi.org/project/your-package/ +``` + +### Why annotated tags? + +Annotated tags (`git tag -a`) carry a tagger identity, date, and message — lightweight tags do +not. `setuptools_scm` works with both, but annotated tags are safer for release governance because +they record *who* created the tag. + +--- + +## 4. Role-Based Access Control + +| Role | What they can do | +|---|---| +| **Maintainer** | Create release tags, approve PRs, manage branch protection | +| **Contributor** | Open PRs to `develop`; cannot push to `main` or create release tags | +| **CI (GitHub Actions)** | Publish to PyPI via OIDC; cannot push code or create tags | + +### Implement via GitHub Teams + +```bash +# Create a Maintainers team and restrict tag creation to that team +gh api repos/{owner}/{repo}/tags/protection \ + --method POST \ + --field pattern="v*" +# Then set allowed actors to the Maintainers team only +``` + +--- + +## 5. Secure Publishing with OIDC (Trusted Publishing) + +**Never store a PyPI API token as a GitHub secret.** Use Trusted Publishing (OIDC) instead. +The PyPI project authorises a specific GitHub repository + workflow + environment — no long-lived +secret is exchanged. + +### One-time PyPI Setup + +1. Go to https://pypi.org/manage/project/your-package/settings/publishing/ +2. Click **Add a new publisher** +3. Fill in: + - **Owner:** your-github-username + - **Repository:** your-repo-name + - **Workflow name:** `publish.yml` + - **Environment name:** `release` (must match the `environment:` key in the workflow) +4. Save. No token required. + +### GitHub Environment Setup + +1. Go to **GitHub → Settings → Environments → New environment** → name it `release` +2. Add a protection rule: **Required reviewers** (optional but recommended for extra safety) +3. Add a deployment branch rule: **Only tags matching `v*`** + +### Minimal `publish.yml` using OIDC + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v[0-9]+.[0-9]+.[0-9]+*" # Matches v1.0.0, v2.0.0a1, v1.2.3rc1 + +jobs: + publish: + name: Build and publish + runs-on: ubuntu-latest + environment: release # Must match the PyPI Trusted Publisher environment name + permissions: + id-token: write # Required for OIDC — grants a short-lived token to PyPI + contents: read + + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED for setuptools_scm + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build + run: pip install build + + - name: Build distributions + run: python -m build + + - name: Validate distributions + run: pip install twine ; twine check dist/* + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + # No `password:` or `user:` needed — OIDC handles authentication +``` + +--- + +## 6. Validate Tag Author in CI + +Restrict who can trigger a release by checking `GITHUB_ACTOR` against an allowlist. +Add this as the **first step** in your publish job to fail fast. + +```yaml +- name: Validate tag author + run: | + ALLOWED_USERS=("your-github-username" "co-maintainer-username") + if [[ ! " ${ALLOWED_USERS[*]} " =~ " ${GITHUB_ACTOR} " ]]; then + echo "::error::Release blocked: ${GITHUB_ACTOR} is not an authorised releaser." + exit 1 + fi + echo "Release authorised for ${GITHUB_ACTOR}." +``` + +### Notes + +- `GITHUB_ACTOR` is the GitHub username of the person who pushed the tag. +- Store the allowlist in a separate file (e.g., `.github/MAINTAINERS`) for maintainability. +- For teams: replace the username check with a GitHub API call to verify team membership. + +--- + +## 7. Prevent Invalid Release Tags + +Reject workflow runs triggered by tags that do not follow your versioning convention. +This stops accidental publishes from tags like `test`, `backup-old`, or `v1`. + +```yaml +- name: Validate release tag format + run: | + # Accepts: v1.0.0 v1.0.0a1 v1.0.0b2 v1.0.0rc1 v1.0.0.post1 + if [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(a|b|rc|\.post)[0-9]*$ ]] && \ + [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::Tag '${GITHUB_REF}' does not match the required format v..[pre]." + exit 1 + fi + echo "Tag format valid: ${GITHUB_REF}" +``` + +### Regex explained + +| Pattern | Matches | +|---|---| +| `v[0-9]+\.[0-9]+\.[0-9]+` | `v1.0.0`, `v12.3.4` | +| `(a\|b\|rc)[0-9]*` | `v1.0.0a1`, `v2.0.0rc2` | +| `\.post[0-9]*` | `v1.0.0.post1` | + +--- + +## 8. Full `publish.yml` with Governance Gates + +Complete workflow combining tag validation, author check, TestPyPI gate, and production publish. + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v[0-9]+.[0-9]+.[0-9]+*" + +jobs: + publish: + name: Build, validate, and publish + runs-on: ubuntu-latest + environment: release + permissions: + id-token: write + contents: read + + steps: + - name: Validate release tag format + run: | + if [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(a[0-9]*|b[0-9]*|rc[0-9]*|\.post[0-9]*)?$ ]]; then + echo "::error::Invalid tag format: ${GITHUB_REF}" + exit 1 + fi + + - name: Validate tag author + run: | + ALLOWED_USERS=("your-github-username") + if [[ ! " ${ALLOWED_USERS[*]} " =~ " ${GITHUB_ACTOR} " ]]; then + echo "::error::${GITHUB_ACTOR} is not authorised to release." + exit 1 + fi + + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build tooling + run: pip install build twine + + - name: Build + run: python -m build + + - name: Validate distributions + run: twine check dist/* + + - name: Publish to TestPyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + continue-on-error: true # Non-fatal; remove if you always want this to pass + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +``` + +### Security checklist + +- [ ] PyPI Trusted Publishing configured (no API token stored in GitHub) +- [ ] GitHub `release` environment has branch protection: tags matching `v*` only +- [ ] Tag format validation step is the first step in the job +- [ ] Allowed-users list is maintained and reviewed regularly +- [ ] No secrets printed in logs (check all `echo` and `run` steps) +- [ ] `permissions:` is scoped to `id-token: write` only — no `write-all` diff --git a/skills/python-pypi-package-builder/references/testing-quality.md b/skills/python-pypi-package-builder/references/testing-quality.md new file mode 100644 index 000000000..b6fbbec25 --- /dev/null +++ b/skills/python-pypi-package-builder/references/testing-quality.md @@ -0,0 +1,257 @@ +# Testing and Code Quality + +## Table of Contents +1. [conftest.py](#1-conftestpy) +2. [Unit tests](#2-unit-tests) +3. [Backend unit tests](#3-backend-unit-tests) +4. [Running tests](#4-running-tests) +5. [Code quality tools](#5-code-quality-tools) +6. [Pre-commit hooks](#6-pre-commit-hooks) + +--- + +## 1. `conftest.py` + +Use `conftest.py` to define shared fixtures. Keep fixtures focused — one fixture per concern. +For async tests, use `pytest-asyncio` with `asyncio_mode = "auto"` in `pyproject.toml`. + +```python +# tests/conftest.py +import pytest +from your_package.core import YourClient +from your_package.backends.memory import MemoryBackend + + +@pytest.fixture +def memory_backend() -> MemoryBackend: + return MemoryBackend() + + +@pytest.fixture +def client(memory_backend: MemoryBackend) -> YourClient: + return YourClient( + api_key="test-key", + backend=memory_backend, + ) +``` + +--- + +## 2. Unit Tests + +Test both the happy path and the edge cases (e.g. invalid inputs, error conditions). + +```python +# tests/test_core.py +import pytest +from your_package import YourClient +from your_package.exceptions import YourPackageError + + +def test_client_creates_with_valid_key(): + client = YourClient(api_key="sk-test") + assert client is not None + + +def test_client_raises_on_empty_key(): + with pytest.raises(ValueError, match="api_key"): + YourClient(api_key="") + + +def test_client_raises_on_invalid_timeout(): + with pytest.raises(ValueError, match="timeout"): + YourClient(api_key="sk-test", timeout=-1) + + +@pytest.mark.asyncio +async def test_process_returns_expected_result(client: YourClient): + result = await client.process({"input": "value"}) + assert "output" in result + + +@pytest.mark.asyncio +async def test_process_raises_on_invalid_input(client: YourClient): + with pytest.raises(YourPackageError): + await client.process({}) # empty input should fail +``` + +--- + +## 3. Backend Unit Tests + +Test each backend independently, in isolation from the rest of the library. This makes failures +easier to diagnose and ensures your abstract interface is actually implemented correctly. + +```python +# tests/test_backends.py +import pytest +from your_package.backends.memory import MemoryBackend + + +@pytest.mark.asyncio +async def test_set_and_get(): + backend = MemoryBackend() + await backend.set("key1", "value1") + result = await backend.get("key1") + assert result == "value1" + + +@pytest.mark.asyncio +async def test_get_missing_key_returns_none(): + backend = MemoryBackend() + result = await backend.get("nonexistent") + assert result is None + + +@pytest.mark.asyncio +async def test_delete_removes_key(): + backend = MemoryBackend() + await backend.set("key1", "value1") + await backend.delete("key1") + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_ttl_expires_entry(): + import asyncio + backend = MemoryBackend() + await backend.set("key1", "value1", ttl=1) + await asyncio.sleep(1.1) + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_different_keys_are_independent(): + backend = MemoryBackend() + await backend.set("key1", "a") + await backend.set("key2", "b") + assert await backend.get("key1") == "a" + assert await backend.get("key2") == "b" + await backend.delete("key1") + assert await backend.get("key2") == "b" +``` + +--- + +## 4. Running Tests + +```bash +pip install -e ".[dev]" +pytest # All tests +pytest --cov --cov-report=html # With HTML coverage report (opens in browser) +pytest -k "test_middleware" # Filter by name +pytest -x # Stop on first failure +pytest -v # Verbose output +``` + +Coverage config in `pyproject.toml` enforces a minimum threshold (`fail_under = 80`). CI will +fail if you drop below it, which catches coverage regressions automatically. + +--- + +## 5. Code Quality Tools + +### Ruff (linting — replaces flake8, pylint, many others) + +```bash +pip install ruff +ruff check . # Check for issues +ruff check . --fix # Auto-fix safe issues +``` + +Ruff is extremely fast and replaces most of the Python linting ecosystem. Configure it in +`pyproject.toml` — see `references/pyproject-toml.md` for the full config. + +### Black (formatting) + +```bash +pip install black +black . # Format all files +black . --check # CI mode — reports issues without modifying files +``` + +### isort (import sorting) + +```bash +pip install isort +isort . # Sort imports +isort . --check-only # CI mode +``` + +Always set `profile = "black"` in `[tool.isort]` — otherwise black and isort conflict. + +### mypy (static type checking) + +```bash +pip install mypy +mypy your_package/ # Type-check your package source only +``` + +Common fixes: + +- `ignore_missing_imports = true` — ignore untyped third-party deps +- `from __future__ import annotations` — enables PEP 563 deferred evaluation (Python 3.9 compat) +- `pip install types-redis` — type stubs for the redis library + +### Run all at once + +```bash +ruff check . && black . --check && isort . --check-only && mypy your_package/ +``` + +--- + +## 6. Pre-commit Hooks + +Pre-commit runs all quality tools automatically before each commit, so issues never reach CI. +Install once per clone with `pre-commit install`. + +```yaml +# .pre-commit-config.yaml +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 + hooks: + - id: ruff + args: [--fix] + - id: ruff-format + + - repo: https://github.com/psf/black + rev: 24.4.2 + hooks: + - id: black + + - repo: https://github.com/pycqa/isort + rev: 5.13.2 + hooks: + - id: isort + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + additional_dependencies: [types-redis] # Add stubs for typed dependencies + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-toml + - id: check-merge-conflict + - id: debug-statements + - id: no-commit-to-branch + args: [--branch, master, --branch, main] +``` + +```bash +pip install pre-commit +pre-commit install # Install once per clone +pre-commit run --all-files # Run all hooks manually (useful before the first install) +``` + +The `no-commit-to-branch` hook prevents accidentally committing directly to `main`/`master`, +which would bypass CI checks. Always work on a feature branch. diff --git a/skills/python-pypi-package-builder/references/tooling-ruff.md b/skills/python-pypi-package-builder/references/tooling-ruff.md new file mode 100644 index 000000000..1d3cc27a8 --- /dev/null +++ b/skills/python-pypi-package-builder/references/tooling-ruff.md @@ -0,0 +1,344 @@ +# Tooling — Ruff-Only Setup and Code Quality + +## Table of Contents +1. [Use Only Ruff (Replaces black, isort, flake8)](#1-use-only-ruff-replaces-black-isort-flake8) +2. [Ruff Configuration in pyproject.toml](#2-ruff-configuration-in-pyprojecttoml) +3. [mypy Configuration](#3-mypy-configuration) +4. [pre-commit Configuration](#4-pre-commit-configuration) +5. [pytest and Coverage Configuration](#5-pytest-and-coverage-configuration) +6. [Dev Dependencies in pyproject.toml](#6-dev-dependencies-in-pyprojecttoml) +7. [CI Lint Job — Ruff Only](#7-ci-lint-job--ruff-only) +8. [Migration Guide — Removing black and isort](#8-migration-guide--removing-black-and-isort) + +--- + +## 1. Use Only Ruff (Replaces black, isort, flake8) + +**Decision:** Use `ruff` as the single linting and formatting tool. Remove `black` and `isort`. + +| Old (avoid) | New (use) | What it does | +|---|---|---| +| `black` | `ruff format` | Code formatting | +| `isort` | `ruff check --select I` | Import sorting | +| `flake8` | `ruff check` | Style and error linting | +| `pyupgrade` | `ruff check --select UP` | Upgrade syntax to modern Python | +| `bandit` | `ruff check --select S` | Security linting | +| All of the above | `ruff` | One tool, one config section | + +**Why ruff?** +- 10–100× faster than the tools it replaces (written in Rust). +- Single config section in `pyproject.toml` — no `.flake8`, `.isort.cfg`, `pyproject.toml[tool.black]` sprawl. +- Actively maintained by Astral; follows the same rules as the tools it replaces. +- `ruff format` is black-compatible — existing black-formatted code passes without changes. + +--- + +## 2. Ruff Configuration in pyproject.toml + +```toml +[tool.ruff] +target-version = "py310" # Minimum supported Python version +line-length = 88 # black-compatible default +src = ["src", "tests"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # pyflakes + "I", # isort + "B", # flake8-bugbear (opinionated but very useful) + "C4", # flake8-comprehensions + "UP", # pyupgrade (modernise syntax) + "SIM", # flake8-simplify + "TCH", # flake8-type-checking (move imports to TYPE_CHECKING block) + "ANN", # flake8-annotations (enforce type hints — remove if too strict) + "S", # flake8-bandit (security) + "N", # pep8-naming +] +ignore = [ + "ANN101", # Missing type annotation for `self` + "ANN102", # Missing type annotation for `cls` + "S101", # Use of `assert` — necessary in tests + "S603", # subprocess without shell=True — often intentional + "B008", # Do not perform function calls in default arguments (false positives in FastAPI/Typer) +] + +[tool.ruff.lint.isort] +known-first-party = ["your_package"] + +[tool.ruff.lint.per-file-ignores] +"tests/**" = ["S101", "ANN", "D"] # Allow assert and skip annotations/docstrings in tests + +[tool.ruff.format] +quote-style = "double" # black-compatible +indent-style = "space" +skip-magic-trailing-comma = false +line-ending = "auto" +``` + +### Useful ruff commands + +```bash +# Check for lint issues (no changes) +ruff check . + +# Auto-fix fixable issues +ruff check --fix . + +# Format code (replaces black) +ruff format . + +# Check formatting without changing files (CI mode) +ruff format --check . + +# Run both lint and format check in one command (for CI) +ruff check . && ruff format --check . +``` + +--- + +## 3. mypy Configuration + +```toml +[tool.mypy] +python_version = "3.10" +strict = true +warn_return_any = true +warn_unused_ignores = true +warn_redundant_casts = true +disallow_untyped_defs = true +disallow_incomplete_defs = true +check_untyped_defs = true +no_implicit_optional = true +show_error_codes = true + +# Ignore missing stubs for third-party packages that don't ship types +[[tool.mypy.overrides]] +module = ["redis.*", "pydantic_settings.*"] +ignore_missing_imports = true +``` + +### Running mypy — handle both src and flat layouts + +```bash +# src layout: +mypy src/your_package/ + +# flat layout: +mypy your_package/ +``` + +In CI, detect layout dynamically: + +```yaml +- name: Run mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy your_package/ + fi +``` + +--- + +## 4. pre-commit Configuration + +```yaml +# .pre-commit-config.yaml +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 # Pin to a specific release; update periodically with `pre-commit autoupdate` + hooks: + - id: ruff + args: [--fix] # Auto-fix what can be fixed + - id: ruff-format # Format (replaces black hook) + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + additional_dependencies: + - types-requests + - types-redis + # Add stubs for any typed dependency used in your package + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-toml + - id: check-yaml + - id: check-merge-conflict + - id: check-added-large-files + args: ["--maxkb=500"] +``` + +### ❌ Remove these hooks (replaced by ruff) + +```yaml +# DELETE or never add: +- repo: https://github.com/psf/black # replaced by ruff-format +- repo: https://github.com/PyCQA/isort # replaced by ruff lint I rules +- repo: https://github.com/PyCQA/flake8 # replaced by ruff check +- repo: https://github.com/PyCQA/autoflake # replaced by ruff check F401 +``` + +### Setup + +```bash +pip install pre-commit +pre-commit install # Installs git hook — runs on every commit +pre-commit run --all-files # Run manually on all files +pre-commit autoupdate # Update all hooks to latest pinned versions +``` + +--- + +## 5. pytest and Coverage Configuration + +```toml +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-ra -q --strict-markers --cov=your_package --cov-report=term-missing" +asyncio_mode = "auto" # Enables async tests without @pytest.mark.asyncio decorator + +[tool.coverage.run] +source = ["your_package"] +branch = true +omit = ["**/__main__.py", "**/cli.py"] # omit entry points from coverage + +[tool.coverage.report] +show_missing = true +skip_covered = false +fail_under = 85 # Fail CI if coverage drops below 85% +exclude_lines = [ + "pragma: no cover", + "if TYPE_CHECKING:", + "raise NotImplementedError", + "@abstractmethod", +] +``` + +### asyncio_mode = "auto" — remove @pytest.mark.asyncio + +With `asyncio_mode = "auto"` set in `pyproject.toml`, **do not** add `@pytest.mark.asyncio` +to test functions. The decorator is redundant and will raise a warning in modern pytest-asyncio. + +```python +# WRONG — the decorator is deprecated when asyncio_mode = "auto": +@pytest.mark.asyncio +async def test_async_operation(): + result = await my_async_func() + assert result == expected + +# CORRECT — just use async def: +async def test_async_operation(): + result = await my_async_func() + assert result == expected +``` + +--- + +## 6. Dev Dependencies in pyproject.toml + +Declare all dev/test tools in an `[extras]` group named `dev`. + +```toml +[project.optional-dependencies] +dev = [ + "pytest>=8", + "pytest-asyncio>=0.23", + "pytest-cov>=5", + "ruff>=0.4", + "mypy>=1.10", + "pre-commit>=3.7", + "httpx>=0.27", # If testing HTTP transport + "respx>=0.21", # If mocking httpx in tests +] +redis = [ + "redis>=5", +] +docs = [ + "mkdocs-material>=9", + "mkdocstrings[python]>=0.25", +] +``` + +Install dev dependencies: + +```bash +pip install -e ".[dev]" +pip install -e ".[dev,redis]" # Include optional extras +``` + +--- + +## 7. CI Lint Job — Ruff Only + +Replace the separate `black`, `isort`, and `flake8` steps with a single `ruff` step. + +```yaml +# .github/workflows/ci.yml — lint job +lint: + name: Lint & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install dev dependencies + run: pip install -e ".[dev]" + + # Single step: ruff replaces black + isort + flake8 + - name: ruff lint + run: ruff check . + + - name: ruff format check + run: ruff format --check . + + - name: mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy $(basename $(ls -d */))/ 2>/dev/null || mypy . + fi +``` + +--- + +## 8. Migration Guide — Removing black and isort + +If you are converting an existing project that used `black` and `isort`: + +```bash +# 1. Remove black and isort from dev dependencies +pip uninstall black isort + +# 2. Remove black and isort config sections from pyproject.toml +# [tool.black] ← delete this section +# [tool.isort] ← delete this section + +# 3. Add ruff to dev dependencies (see Section 2 for config) + +# 4. Run ruff format to confirm existing code is already compatible +ruff format --check . +# ruff format is black-compatible; output should be identical + +# 5. Update .pre-commit-config.yaml (see Section 4) +# Remove black and isort hooks; add ruff and ruff-format hooks + +# 6. Update CI (see Section 7) +# Remove black, isort, flake8 steps; add ruff check + ruff format --check + +# 7. Reinstall pre-commit hooks +pre-commit uninstall +pre-commit install +pre-commit run --all-files # Verify clean +``` diff --git a/skills/python-pypi-package-builder/references/versioning-strategy.md b/skills/python-pypi-package-builder/references/versioning-strategy.md new file mode 100644 index 000000000..5f49ad8c2 --- /dev/null +++ b/skills/python-pypi-package-builder/references/versioning-strategy.md @@ -0,0 +1,375 @@ +# Versioning Strategy — PEP 440, SemVer, and Decision Engine + +## Table of Contents +1. [PEP 440 — The Standard](#1-pep-440--the-standard) +2. [Semantic Versioning (SemVer)](#2-semantic-versioning-semver) +3. [Pre-release Identifiers](#3-pre-release-identifiers) +4. [Versioning Decision Engine](#4-versioning-decision-engine) +5. [Dynamic Versioning — setuptools_scm (Recommended)](#5-dynamic-versioning--setuptools_scm-recommended) +6. [Hatchling with hatch-vcs Plugin](#6-hatchling-with-hatch-vcs-plugin) +7. [Static Versioning — flit](#7-static-versioning--flit) +8. [Static Versioning — hatchling manual](#8-static-versioning--hatchling-manual) +9. [DO NOT Hardcode Version (except flit)](#9-do-not-hardcode-version-except-flit) +10. [Dependency Version Specifiers](#10-dependency-version-specifiers) +11. [PyPA Release Commands](#11-pypa-release-commands) + +--- + +## 1. PEP 440 — The Standard + +All Python package versions must comply with [PEP 440](https://peps.python.org/pep-0440/). +Non-compliant versions (e.g., `1.0-beta`, `2023.1.1.dev`) will be rejected by PyPI. + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +1.0.0 Stable release +1.0.0a1 Alpha pre-release +1.0.0b2 Beta pre-release +1.0.0rc1 Release candidate +1.0.0.post1 Post-release (packaging fix; same codebase) +1.0.0.dev1 Development snapshot — DO NOT upload to PyPI +2.0.0 Major release (breaking changes) +``` + +### Epoch prefix (rare) + +``` +1!1.0.0 Epoch 1; used when you need to skip ahead of an old scheme +``` + +Use epochs only as a last resort to fix a broken version sequence. + +--- + +## 2. Semantic Versioning (SemVer) + +SemVer maps cleanly onto PEP 440. Always use `MAJOR.MINOR.PATCH`: + +``` +MAJOR Increment when you make incompatible API changes (rename, remove, break) +MINOR Increment when you add functionality backward-compatibly (new features) +PATCH Increment when you make backward-compatible bug fixes + +Examples: + 1.0.0 → 1.0.1 Bug fix, no API change + 1.0.0 → 1.1.0 New method added; existing API intact + 1.0.0 → 2.0.0 Public method renamed or removed +``` + +### What counts as a breaking change? + +| Change | Breaking? | +|---|---| +| Rename a public function | YES — `MAJOR` | +| Remove a parameter | YES — `MAJOR` | +| Add a required parameter | YES — `MAJOR` | +| Add an optional parameter with a default | NO — `MINOR` | +| Add a new function/class | NO — `MINOR` | +| Fix a bug | NO — `PATCH` | +| Update a dependency lower bound | NO (usually) — `PATCH` | +| Update a dependency upper bound (breaking) | YES — `MAJOR` | + +--- + +## 3. Pre-release Identifiers + +Use pre-release versions to get user feedback before a stable release. +Pre-releases are **not** installed by default by pip (`pip install pkg` skips them). +Users must opt-in: `pip install "pkg==2.0.0a1"` or `pip install --pre pkg`. + +``` +1.0.0a1 Alpha-1: very early; expect bugs; API may change +1.0.0b1 Beta-1: feature-complete; API stabilising; seek broader feedback +1.0.0rc1 Release candidate: code-frozen; final testing before stable +1.0.0 Stable: ready for production +``` + +### Increment rule + +``` +Start: 1.0.0a1 +More alphas: 1.0.0a2, 1.0.0a3 +Move to beta: 1.0.0b1 (reset counter) +Move to RC: 1.0.0rc1 +Stable: 1.0.0 +``` + +--- + +## 4. Versioning Decision Engine + +Use this decision tree to pick the right versioning strategy before writing any code. + +``` +Is the project using git and tagging releases with version tags? +├── YES → setuptools + setuptools_scm (DEFAULT — best for most projects) +│ Git tag v1.0.0 becomes the installed version automatically. +│ Zero manual version bumping. +│ +└── NO — Is the project a simple, single-module library with infrequent releases? + ├── YES → flit + │ Set __version__ = "1.0.0" in __init__.py. + │ Update manually before each release. + │ + └── NO — Does the team want an integrated build + dep management tool? + ├── YES → poetry + │ Manage version in [tool.poetry] version field. + │ + └── NO → hatchling (modern, fast, pure-Python) + Use hatch-vcs plugin for dynamic versioning + or set version manually in [project]. + +Does the package have C/Cython/Fortran extensions? +└── YES (always) → setuptools (only backend with native extension support) +``` + +### Summary Table + +| Backend | Version source | Best for | +|---|---|---| +| `setuptools` + `setuptools_scm` | Git tags — fully automatic | DEFAULT for new projects | +| `hatchling` + `hatch-vcs` | Git tags — automatic via plugin | hatchling users | +| `flit` | `__version__` in `__init__.py` | Very simple, minimal config | +| `poetry` | `[tool.poetry] version` field | Integrated dep + build management | +| `hatchling` manual | `[project] version` field | One-off static versioning | + +--- + +## 5. Dynamic Versioning — setuptools_scm (Recommended) + +`setuptools_scm` reads the current git tag and computes the version at build time. +No separate `__version__` update step — just tag and push. + +### `pyproject.toml` configuration + +```toml +[build-system] +requires = ["setuptools>=70", "setuptools_scm>=8"] +build-backend = "setuptools.backends.legacy:build" + +[project] +name = "your-package" +dynamic = ["version"] + +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +g from breaking PyPI +``` + +### `__init__.py` — correct version access + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + # Package is not installed (running from a source checkout without pip install -e .) + __version__ = "0.0.0.dev0" + +__all__ = ["__version__"] +``` + +### How the version is computed + +``` +git tag v1.0.0 → installed_version = "1.0.0" +3 commits after v1.0.0 → installed_version = "1.0.0.post3+g" (dev only) +git tag v1.1.0 → installed_version = "1.1.0" +``` + +With `local_scheme = "no-local-version"`, the `+g` suffix is stripped for PyPI +uploads while still being visible locally. + +### Critical CI requirement + +```yaml +- uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED — without this, git has no tag history + # setuptools_scm falls back to 0.0.0+d silently +``` + +**Every** CI job that installs or builds the package must have `fetch-depth: 0`. + +### Debugging version issues + +```bash +# Check what version setuptools_scm would produce right now: +python -m setuptools_scm + +# If you see 0.0.0+d... it means: +# 1. No tags reachable from HEAD, OR +# 2. fetch-depth: 0 was not set in CI +``` + +--- + +## 6. Hatchling with hatch-vcs Plugin + +An alternative to setuptools_scm for teams already using hatchling. + +```toml +[build-system] +requires = ["hatchling", "hatch-vcs"] +build-backend = "hatchling.build" + +[project] +name = "your-package" +dynamic = ["version"] + +[tool.hatch.version] +source = "vcs" + +[tool.hatch.build.hooks.vcs] +version-file = "src/your_package/_version.py" +``` + +Access the version the same way as setuptools_scm: + +```python +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0.dev0" +``` + +--- + +## 7. Static Versioning — flit + +Use flit only for simple, single-module packages where manual version bumping is acceptable. + +### `pyproject.toml` + +```toml +[build-system] +requires = ["flit_core>=3.9"] +build-backend = "flit_core.buildapi" + +[project] +name = "your-package" +dynamic = ["version", "description"] +``` + +### `__init__.py` + +```python +"""your-package — a focused, single-purpose utility.""" +__version__ = "1.2.0" # flit reads this; update manually before each release +``` + +**flit exception:** this is the ONLY case where hardcoding `__version__` is correct. +flit discovers the version by importing `__init__.py` and reading `__version__`. + +### Release flow for flit + +```bash +# 1. Bump __version__ in __init__.py +# 2. Update CHANGELOG.md +# 3. Commit +git add src/your_package/__init__.py CHANGELOG.md +git commit -m "chore: release v1.2.0" +# 4. Tag (flit can also publish directly) +git tag v1.2.0 +git push origin v1.2.0 +# 5. Build and publish +flit publish +# OR +python -m build && twine upload dist/* +``` + +--- + +## 8. Static Versioning — hatchling manual + +```toml +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "your-package" +version = "1.0.0" # Manual; update before each release +``` + +Update `version` in `pyproject.toml` before every release. No `__version__` required +(access via `importlib.metadata.version()` as usual). + +--- + +## 9. DO NOT Hardcode Version (except flit) + +Hardcoding `__version__` in `__init__.py` when **not** using flit creates a dual source of +truth that diverges over time. + +```python +# BAD — when using setuptools_scm, hatchling, or poetry: +__version__ = "1.0.0" # gets stale; diverges from the installed package version + +# GOOD — works for all backends except flit: +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0.dev0" +``` + +--- + +## 10. Dependency Version Specifiers + +Pick the right specifier style to avoid poisoning your users' environments. + +```toml +# [project] dependencies — library best practices: + +"httpx>=0.24" # Minimum only — PREFERRED; lets users upgrade freely +"httpx>=0.24,<2.0" # Upper bound only when a known breaking change exists in next major +"requests>=2.28,<3.0" # Acceptable for well-known major-version breaks + +# Application / CLI (pinning is fine): +"httpx==0.27.2" # Lock exact version for reproducible deploys + +# NEVER in a library: +# "httpx~=0.24.0" # Too tight; blocks minor upgrades +# "httpx==0.27.*" # Not valid PEP 440 +# "httpx" # No constraint; fragile against future breakage +``` + +--- + +## 11. PyPA Release Commands + +The canonical sequence from code to user install. + +```bash +# Step 1: Tag the release (triggers CI publish.yml automatically if configured) +git tag -a v1.2.3 -m "Release v1.2.3" +git push origin v1.2.3 + +# Step 2 (manual fallback only): Build locally +python -m build +# Produces: +# dist/your_package-1.2.3.tar.gz (sdist) +# dist/your_package-1.2.3-py3-none-any.whl (wheel) + +# Step 3: Validate +twine check dist/* + +# Step 4: Test on TestPyPI first (first release or major change) +twine upload --repository testpypi dist/* +pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ your-package==1.2.3 + +# Step 5: Publish to production PyPI +twine upload dist/* +# OR via GitHub Actions (recommended): +# push the tag → publish.yml runs → pypa/gh-action-pypi-publish handles upload via OIDC + +# Step 6: Verify +pip install your-package==1.2.3 +python -c "import your_package; print(your_package.__version__)" +``` diff --git a/skills/python-pypi-package-builder/scripts/scaffold.py b/skills/python-pypi-package-builder/scripts/scaffold.py new file mode 100644 index 000000000..4479d8314 --- /dev/null +++ b/skills/python-pypi-package-builder/scripts/scaffold.py @@ -0,0 +1,920 @@ +#!/usr/bin/env python3 +""" +scaffold.py — Generate a production-grade Python PyPI package structure. + +Usage: + python scaffold.py --name my-package + python scaffold.py --name my-package --layout src + python scaffold.py --name my-package --build hatchling + +Options: + --name PyPI package name (lowercase, hyphens). Required. + --layout 'flat' (default) or 'src'. + --build 'setuptools' (default, uses setuptools_scm) or 'hatchling'. + --author Author name (default: Your Name). + --email Author email (default: you@example.com). + --output Output directory (default: current directory). +""" + +import argparse +import os +import sys +import textwrap +from pathlib import Path + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def pkg_name(pypi_name: str) -> str: + """Convert 'my-pkg' → 'my_pkg'.""" + return pypi_name.replace("-", "_") + + +def write(path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(textwrap.dedent(content).lstrip(), encoding="utf-8") + print(f" created {path}") + + +def touch(path: Path) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.touch() + print(f" created {path}") + + +# --------------------------------------------------------------------------- +# File generators +# --------------------------------------------------------------------------- + +def gen_pyproject_setuptools(name: str, mod: str, author: str, email: str, layout: str) -> str: + packages_find = ( + 'where = ["src"]' if layout == "src" else f'include = ["{mod}*"]' + ) + pkg_data_key = f"src/{mod}" if layout == "src" else mod + pythonpath = "" if layout == "src" else '\npythonpath = ["."]' + return f'''\ +[build-system] +requires = ["setuptools>=68", "wheel", "setuptools_scm"] +build-backend = "setuptools.build_meta" + +[project] +name = "{name}" +dynamic = ["version"] +description = "" +readme = "README.md" +requires-python = ">=3.10" +license = {{text = "MIT"}} +authors = [ + {{name = "{author}", email = "{email}"}}, +] +keywords = [ + "python", + # Add 10-15 specific keywords — they affect PyPI discoverability +] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Topic :: Software Development :: Libraries :: Python Modules", + "Typing :: Typed", +] +dependencies = [ + # List your runtime dependencies here. Keep them minimal. + # Example: "httpx>=0.24", "pydantic>=2.0" +] +] + +[project.optional-dependencies] +redis = [ + "redis>=4.2", +] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/{name}" +Documentation = "https://github.com/yourusername/{name}#readme" +Repository = "https://github.com/yourusername/{name}" +"Bug Tracker" = "https://github.com/yourusername/{name}/issues" +Changelog = "https://github.com/yourusername/{name}/blob/master/CHANGELOG.md" + +[tool.setuptools.packages.find] +{packages_find} + +[tool.setuptools.package-data] +{mod} = ["py.typed"] + +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" + +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "N", "UP", "B", "SIM", "C4", "PTH"] + +[tool.ruff.lint.per-file-ignores] +"tests/*" = ["S101"] + +[tool.black] +line-length = 100 +target-version = ["py310", "py311", "py312", "py313"] + +[tool.isort] +profile = "black" +line_length = 100 + +[tool.mypy] +python_version = "3.10" +warn_return_any = true +warn_unused_configs = true +disallow_untyped_defs = true +ignore_missing_imports = true +strict = false + +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"]{pythonpath} +python_files = "test_*.py" +python_classes = "Test*" +python_functions = "test_*" +addopts = "-v --tb=short --cov={mod} --cov-report=term-missing" + +[tool.coverage.run] +source = ["{mod}"] +omit = ["tests/*"] + +[tool.coverage.report] +fail_under = 80 +show_missing = true +''' + + +def gen_pyproject_hatchling(name: str, mod: str, author: str, email: str) -> str: + return f'''\ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "{name}" +version = "0.1.0" +description = "" +readme = "README.md" +requires-python = ">=3.10" +license = {{text = "MIT"}} +authors = [ + {{name = "{author}", email = "{email}"}}, +] +keywords = ["python"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [ + # List your runtime dependencies here. +] + +[project.optional-dependencies] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/{name}" +Changelog = "https://github.com/yourusername/{name}/blob/master/CHANGELOG.md" + +[tool.hatch.build.targets.wheel] +packages = ["{mod}"] + +[tool.hatch.build.targets.wheel.sources] +"{mod}" = "{mod}" + +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.black] +line-length = 100 + +[tool.isort] +profile = "black" + +[tool.mypy] +python_version = "3.10" +disallow_untyped_defs = true +ignore_missing_imports = true + +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"] +addopts = "-v --tb=short --cov={mod} --cov-report=term-missing" + +[tool.coverage.report] +fail_under = 80 +show_missing = true +''' + + +def gen_init(name: str, mod: str) -> str: + return f'''\ +"""{name}: .""" + +from importlib.metadata import PackageNotFoundError, version + +try: + __version__ = version("{name}") +except PackageNotFoundError: + __version__ = "0.0.0-dev" + +from {mod}.core import YourClient +from {mod}.config import YourSettings +from {mod}.exceptions import YourPackageError + +__all__ = [ + "YourClient", + "YourSettings", + "YourPackageError", + "__version__", +] +''' + + +def gen_core(mod: str) -> str: + return f'''\ +from __future__ import annotations + +from {mod}.exceptions import YourPackageError + + +class YourClient: + """ + Main entry point for . + + Args: + api_key: Required authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from {mod} import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process(data) + """ + + def __init__( + self, + api_key: str, + timeout: int = 30, + retries: int = 3, + ) -> None: + if not api_key: + raise ValueError("api_key must not be empty") + if timeout <= 0: + raise ValueError("timeout must be positive") + self._api_key = api_key + self.timeout = timeout + self.retries = retries + + def process(self, data: dict) -> dict: + """ + Process data and return results. + + Args: + data: Input dictionary to process. + + Returns: + Processed result as a dictionary. + + Raises: + YourPackageError: If processing fails. + """ + raise NotImplementedError +''' + + +def gen_exceptions(mod: str) -> str: + return f'''\ +class YourPackageError(Exception): + """Base exception for {mod}.""" + + +class YourPackageConfigError(YourPackageError): + """Raised on invalid configuration.""" +''' + + +def gen_backends_init() -> str: + return '''\ +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage backend interface.""" + + @abstractmethod + async def get(self, key: str) -> str | None: + """Retrieve a value by key. Returns None if not found.""" + ... + + @abstractmethod + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value. Optional TTL in seconds.""" + ... + + @abstractmethod + async def delete(self, key: str) -> None: + """Delete a key.""" + ... +''' + + +def gen_memory_backend() -> str: + return '''\ +from __future__ import annotations + +import asyncio +import time + +from . import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. Zero extra dependencies.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = asyncio.Lock() + + async def get(self, key: str) -> str | None: + async with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.time() > expires_at: + del self._store[key] + return None + return value + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + async with self._lock: + expires_at = time.time() + ttl if ttl is not None else None + self._store[key] = (value, expires_at) + + async def delete(self, key: str) -> None: + async with self._lock: + self._store.pop(key, None) +''' + + +def gen_conftest(name: str, mod: str) -> str: + return f'''\ +import pytest + +from {mod}.backends.memory import MemoryBackend +from {mod}.core import YourClient + + +@pytest.fixture +def memory_backend() -> MemoryBackend: + return MemoryBackend() + + +@pytest.fixture +def client(memory_backend: MemoryBackend) -> YourClient: + return YourClient( + api_key="test-key", + backend=memory_backend, + ) +''' + + +def gen_test_core(mod: str) -> str: + return f'''\ +import pytest + +from {mod} import YourClient +from {mod}.exceptions import YourPackageError + + +def test_client_creates_with_valid_key() -> None: + client = YourClient(api_key="sk-test") + assert client is not None + + +def test_client_raises_on_empty_key() -> None: + with pytest.raises(ValueError, match="api_key"): + YourClient(api_key="") + + +def test_client_raises_on_invalid_timeout() -> None: + with pytest.raises(ValueError, match="timeout"): + YourClient(api_key="sk-test", timeout=-1) +''' + + +def gen_test_backends() -> str: + return '''\ +import pytest +from your_package.backends.memory import MemoryBackend + + +@pytest.mark.asyncio +async def test_set_and_get() -> None: + backend = MemoryBackend() + await backend.set("key1", "value1") + result = await backend.get("key1") + assert result == "value1" + + +@pytest.mark.asyncio +async def test_get_missing_key_returns_none() -> None: + backend = MemoryBackend() + result = await backend.get("nonexistent") + assert result is None + + +@pytest.mark.asyncio +async def test_delete_removes_key() -> None: + backend = MemoryBackend() + await backend.set("key1", "value1") + await backend.delete("key1") + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_different_keys_are_independent() -> None: + backend = MemoryBackend() + await backend.set("key1", "a") + await backend.set("key2", "b") + assert await backend.get("key1") == "a" + assert await backend.get("key2") == "b" +''' + + +def gen_ci_yml(name: str, mod: str) -> str: + return f'''\ +name: CI + +on: + push: + branches: [main, master] + pull_request: + branches: [main, master] + +jobs: + lint: + name: Lint, Format & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install dev dependencies + run: pip install -e ".[dev]" + - name: ruff + run: ruff check . + - name: black + run: black . --check + - name: isort + run: isort . --check-only + - name: mypy + run: mypy {mod}/ + + test: + name: Test (Python ${{{{ matrix.python-version }}}}) + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10", "3.11", "3.12", "3.13"] + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@v5 + with: + python-version: ${{{{ matrix.python-version }}}} + - name: Install dependencies + run: pip install -e ".[dev]" + - name: Run tests with coverage + run: pytest --cov --cov-report=xml + - name: Upload coverage + uses: codecov/codecov-action@v4 + with: + fail_ci_if_error: false +''' + + +def gen_publish_yml() -> str: + return '''\ +name: Publish to PyPI + +on: + push: + tags: + - "v*.*.*" + +jobs: + build: + name: Build distribution + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install build tools + run: pip install build twine + - name: Build package + run: python -m build + - name: Check distribution + run: twine check dist/* + - uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish: + name: Publish to PyPI + needs: build + runs-on: ubuntu-latest + environment: pypi + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +''' + + +def gen_precommit() -> str: + return '''\ +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 + hooks: + - id: ruff + args: [--fix] + - id: ruff-format + + - repo: https://github.com/psf/black + rev: 24.4.2 + hooks: + - id: black + + - repo: https://github.com/pycqa/isort + rev: 5.13.2 + hooks: + - id: isort + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-toml + - id: check-merge-conflict + - id: debug-statements + - id: no-commit-to-branch + args: [--branch, master, --branch, main] +''' + + +def gen_changelog(name: str) -> str: + return f'''\ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [Unreleased] + +### Added +- Initial project scaffold + +[Unreleased]: https://github.com/yourusername/{name}/commits/master +''' + + +def gen_readme(name: str, mod: str) -> str: + return f'''\ +# {name} + +> One-line description — what it does and why it's useful. + +[![PyPI version](https://badge.fury.io/py/{name}.svg)](https://pypi.org/project/{name}/) +[![Python Versions](https://img.shields.io/pypi/pyversions/{name})](https://pypi.org/project/{name}/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) + +## Installation + +```bash +pip install {name} +``` + +## Quick Start + +```python +from {mod} import YourClient + +client = YourClient(api_key="sk-...") +result = client.process({{"input": "value"}}) +print(result) +``` + +## Configuration + +| Parameter | Type | Default | Description | +|---|---|---|---| +| api_key | str | required | Authentication credential | +| timeout | int | 30 | Request timeout in seconds | +| retries | int | 3 | Number of retry attempts | + +## Contributing + +See [CONTRIBUTING.md](./CONTRIBUTING.md) + +## Changelog + +See [CHANGELOG.md](./CHANGELOG.md) + +## License + +MIT — see [LICENSE](./LICENSE) +''' + + +def gen_setup_py() -> str: + return '''\ +# Thin shim for legacy editable install compatibility. +# All configuration lives in pyproject.toml. +from setuptools import setup + +setup() +''' + + +def gen_license(author: str) -> str: + return f'''\ +MIT License + +Copyright (c) 2026 {author} + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +''' + + +# --------------------------------------------------------------------------- +# Main scaffold +# --------------------------------------------------------------------------- + +def scaffold( + name: str, + layout: str, + build: str, + author: str, + email: str, + output: str, +) -> None: + mod = pkg_name(name) + root = Path(output) / name + pkg_root = root / "src" / mod if layout == "src" else root / mod + + print(f"\nScaffolding {name!r} ({layout} layout, {build} build backend)\n") + + # Package source + touch(pkg_root / "py.typed") + write(pkg_root / "__init__.py", gen_init(name, mod)) + write(pkg_root / "core.py", gen_core(mod)) + write(pkg_root / "exceptions.py", gen_exceptions(mod)) + write(pkg_root / "backends" / "__init__.py", gen_backends_init()) + write(pkg_root / "backends" / "memory.py", gen_memory_backend()) + + # Tests + write(root / "tests" / "__init__.py", "") + write(root / "tests" / "conftest.py", gen_conftest(name, mod)) + write(root / "tests" / "test_core.py", gen_test_core(mod)) + write(root / "tests" / "test_backends.py", gen_test_backends()) + + # CI + write(root / ".github" / "workflows" / "ci.yml", gen_ci_yml(name, mod)) + write(root / ".github" / "workflows" / "publish.yml", gen_publish_yml()) + write( + root / ".github" / "ISSUE_TEMPLATE" / "bug_report.md", + """\ +--- +name: Bug Report +about: Report a reproducible bug +labels: bug +--- + +**Python version:** +**Package version:** + +**Describe the bug:** + +**Minimal reproducible example:** +```python +# paste here +``` + +**Expected behavior:** + +**Actual behavior:** +""", + ) + write( + root / ".github" / "ISSUE_TEMPLATE" / "feature_request.md", + """\ +--- +name: Feature Request +about: Suggest a new feature +labels: enhancement +--- + +**Problem this would solve:** + +**Proposed solution:** + +**Alternatives considered:** +""", + ) + + # Config files + write(root / ".pre-commit-config.yaml", gen_precommit()) + write(root / "CHANGELOG.md", gen_changelog(name)) + write(root / "README.md", gen_readme(name, mod)) + write(root / "LICENSE", gen_license(author)) + + # pyproject.toml + setup.py + if build == "setuptools": + write(root / "pyproject.toml", gen_pyproject_setuptools(name, mod, author, email, layout)) + write(root / "setup.py", gen_setup_py()) + else: + write(root / "pyproject.toml", gen_pyproject_hatchling(name, mod, author, email)) + + # .gitignore + write( + root / ".gitignore", + """\ +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +build/ +dist/ +*.egg-info/ +.eggs/ +*.egg +.env +.venv +venv/ +.mypy_cache/ +.ruff_cache/ +.pytest_cache/ +htmlcov/ +.coverage +cov_annotate/ +*.xml +""", + ) + + print(f"\nDone! Created {root.resolve()}") + print("\nNext steps:") + print(f" cd {name}") + print(" git init && git add .") + print(' git commit -m "chore: initial scaffold"') + print(" pip install -e '.[dev]'") + print(" pre-commit install") + print(" pytest") + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def main() -> None: + parser = argparse.ArgumentParser( + description="Scaffold a production-grade Python PyPI package." + ) + parser.add_argument( + "--name", + required=True, + help="PyPI package name (lowercase, hyphens). Example: my-package", + ) + parser.add_argument( + "--layout", + choices=["flat", "src"], + default="flat", + help="Project layout: 'flat' (default) or 'src'.", + ) + parser.add_argument( + "--build", + choices=["setuptools", "hatchling"], + default="setuptools", + help="Build backend: 'setuptools' (default, uses setuptools_scm) or 'hatchling'.", + ) + parser.add_argument("--author", default="Your Name", help="Author name.") + parser.add_argument("--email", default="you@example.com", help="Author email.") + parser.add_argument("--output", default=".", help="Output directory (default: .).") + args = parser.parse_args() + + # Validate name + import re + if not re.match(r"^[a-z][a-z0-9\-]*$", args.name): + print( + f"Error: --name must be lowercase letters, digits, and hyphens only. Got: {args.name!r}", + file=sys.stderr, + ) + sys.exit(1) + + target = Path(args.output) / args.name + if target.exists(): + print(f"Error: {target} already exists.", file=sys.stderr) + sys.exit(1) + + scaffold( + name=args.name, + layout=args.layout, + build=args.build, + author=args.author, + email=args.email, + output=args.output, + ) + + +if __name__ == "__main__": + main()