Pin serverless databricks-connect by major (~=17.0, not ~=17.3.0)

[rugpanov]

Problem

For serverless, the dev-group databricks-connect pin resolved too narrowly. dbconnect_pin() took the first two segments of the listed version (17.3.1 → ~=17.3.0), producing a compatible-release pin of >=17.3.0, <17.4.0. That locks databricks-connect to a single minor line and only picks up patch bumps — but a serverless environment version tracks a whole major runtime line, so it should follow the latest release within that major.

Fix

Pin by bare major: ~=MAJOR.0 (e.g. 17.3.1 → databricks-connect~=17.0, resolving >=17.0, <18.0). One-line change in envgen.py; the committed serverless pyprojects are regenerated to match.

env	before	after
serverless-v1	~=14.3.0	~=14.0
serverless-v2	~=15.4.0	~=15.0
serverless-v3	~=16.4.0	~=16.0
serverless-v4	~=17.3.0	~=17.0
serverless-v5	~=18.0.0	~=18.0
serverless-v5-ml	~=18.0.0	~=18.0

The DBR path is unaffected — it passes an explicit MAJOR.MINOR pin derived per point-release page, which is intended there.

Verification

Regenerated serverless output matches the committed files byte-for-byte (no drift), so a future sync.py run won't revert it.

This pull request and its description were written by Isaac.

[anton-107]

Reviewed at f303e33. I'd like to hold this one — I think the loosened pin breaks a cross-repo contract the CLI depends on, and I'm also not convinced the premise is right. Details below; happy to be argued out of the second part.

1. It corrupts the version the CLI reports to users

The CLI reads the pin string verbatim out of the published pyproject.toml (libs/localenv/constraints.go:74, consumed at pipeline.go:179) and derives the version it shows users by taking everything from the first digit onward (versionFromPin, pipeline.go:551). I ran your branch's dbconnect_pin directly and traced it through:

published pin	CLI reports dbconnectVersion
databricks-connect~=17.3.0 (today)	17.3.0 — correct
databricks-connect~=17.0 (this PR)	17.0 — not an installable version

With ~=17.0, uv resolves to the newest 17.x — say 17.3.1 — while the CLI advertises 17.0. Two things make this worse than a cosmetic slip:

• --dry-run never self-corrects. Validate is stubbed out in check mode (pipeline.go:203-211), so the wrong value is the only value a dry-run user ever sees. On a real run validate overwrites it with the installed version (pipeline.go:462-467), so dry-run and real runs disagree — the most confusing possible shape.
• It's in the --output json contract the VS Code extension consumes, not just human-readable output.

The pipeline won't fail, at least: validate only asserts the major matches (pipeline.go:446). So this silently reports wrong data rather than erroring.

2. It reshapes every future sync, not just the six files

dbconnect_pin is the default path for all serverless envs, so this isn't a one-time regeneration — a future listing of 19.0.1 becomes ~=19.0, and so on. Worth stating in the description, since it's a permanent change to the generation rule.

3. I don't think the premise holds

The PR argues a serverless environment version "tracks a whole major runtime line." That's true of the runtime, but it's not what these artifacts are for. Per this repo's own README, the goal is reproducing one runtime snapshot — "matching the exact Python version, databricks-connect version, and transitive dependency set."

Concretely: [tool.uv] constraint-dependencies in serverless-v4 stays pinned at that snapshot's grpcio~=1.67.0, pyarrow~=19.0.1, protobuf~=5.29.4, pandas~=2.2.3. Loosening databricks-connect to any 17.x while its transitive deps remain pinned to the 17.3-era set means the first later 17.x that raises a dependency floor produces an unsatisfiable resolution. This loosens the one package whose version defines the snapshot, while leaving everything it depends on frozen.

Also: ~=14.0 is a wider loosening than the table suggests — it spans 14.0 through 14.3+, and no serverless environment version was ever pinned to 14.0.

What I'd suggest

Mostly I want to understand the motivating problem, since that changes the answer:

• If it's patch-currency (picking up fixes within the minor line), ~=17.3 is the PEP 440 spelling you want — it allows 17.3.x and 17.4 without abandoning the line. Note ~=17.3.0 already allows patch bumps, so if that's the goal this may be a no-op.
• If bare-major really is intended, then versionFromPin on the CLI side needs to stop treating a pin as a version first, and that fix should ship before this. Otherwise there's a window where the CLI reports 17.0 to users.

Either way, this coupling deserves a test on at least one side — the fact that a one-line change to envgen.py can silently alter CLI user-facing output, with nothing failing anywhere, is the real gap here. There's currently no test suite for envgen.py at all, and the pin-string format is an undocumented contract between the two repos.

Reviewed with Claude Code; all claims above were verified by running the branch's dbconnect_pin and reading the CLI consumer.