FEAT: Add Conda release and publish tooling for mssql-python

[Jahnvi Thakkar (jahnvi480)]

Work Item / Issue Reference

AB#47315

Summary

This pull request introduces significant improvements to the conda packaging and release process for the mssql-python project. It adds a new GitHub Actions workflow to automatically audit conda packages on pull requests, ensuring early detection of packaging and dependency issues. Additionally, it introduces a standalone Azure DevOps pipeline for validating and publishing conda packages, decoupled from the PyPI release process. These changes enhance the reliability, security, and maintainability of the conda release workflow.

Conda packaging and release automation:

• Added .github/workflows/conda-audit.yml to automatically build and audit a linux-64 conda package using shipped PyPI wheels on every relevant pull request, running comprehensive tests and a masking-immune binary audit to catch regressions early.
• Introduced OneBranchPipelines/conda-release-pipeline.yml, a standalone Azure DevOps pipeline to validate and publish conda packages. This pipeline enforces strict readiness gates, supports dry-run validation, and cleanly separates the conda release from the PyPI/wheel release process.

Release safety and validation improvements:

• The new conda release pipeline includes a dry-run mode (default) that prints the exact anaconda upload plan without publishing, allowing maintainers to review intended actions before production release.
• When publishing, the pipeline enforces version pinning and provenance checks to ensure only the correct, validated packages are released to Anaconda.org, with clear gates and error messages for misconfigurations.

Final reviewed source: 967b03364c022109d5f617a883e5a94ae649ffdb (12 added files, +3,961/-0 against main). The final follow-up changes three files (+100/-58): staging labels must match <target>_staging_<numeric-build-id>, that namespace cannot be a public target, and the actual generated scope is validated before upload and every promoter/cleanup API entrypoint. The publisher lock still contains all 66 pins/hashes, including anaconda-client; the separate audit lock contains only zstandard. 340 focused source tests and offline PowerShell route checks passed; no live publication was performed.

The release version defaults to 1.15.0, with an explicit override. publishToConda=false remains the default; true enables Official publishing after validation. No manual ADO approval or locking system is added; secret-read permissions are unchanged and the team must serialize publication/recovery. The current-source ADO dry run has not been executed. The hash-pinned public 1.14.0 wheel pair remains an older packaging control, not 1.15.0 qualification; its update awaits verified public 1.15.0 wheel metadata and hashes.

[Copilot]

Pull request overview

This PR adds initial conda packaging assets for mssql-python (and its mssql-python-odbc companion) so the project can be published to the Microsoft-owned microsoft Anaconda channel, alongside the existing PyPI wheel distribution.

Changes:

• Adds conda recipes for mssql-python and mssql-python-odbc that repackage existing wheels (no compilation).
• Adds a local PowerShell harness to build both recipes, create a local channel, and validate import / optional live-connect.
• Adds a draft OneBranch stage to build/test/publish conda artifacts from signed release artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
conda/onebranch-publish-conda-stage.yml	Draft OneBranch stage to build from signed artifacts, gate on smoke tests, and publish to the microsoft Anaconda channel.
conda/mssql-python/meta.yaml	Conda recipe for repackaging the mssql-python wheel and depending on a version-locked mssql-python-odbc.
conda/mssql-python-odbc/meta.yaml	Conda recipe for repackaging the proprietary driver wheel into a companion conda package.
conda/build_and_test_local.ps1	Local “test-before-live” harness to build, index, install, and smoke test the conda packages.

Suppressed comments (1)

conda/onebranch-publish-conda-stage.yml:133

• Same string-vs-boolean condition issue here: quoting the template expression turns it into a string, which can cause the publish step to be skipped unexpectedly even when publishToConda is true.

          - task: PowerShell@2
            displayName: 'Publish to anaconda.org/microsoft'
            condition: and(succeeded(), eq('${{ parameters.publishToConda }}', true))

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 83%

📈 Total Lines Covered: 8220 out of 9874 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.row.py: 77.6%
mssql_python.pybind.ddbc_bindings.cpp: 77.7%
mssql_python.pybind.connection.connection_pool.cpp: 81.8%
mssql_python.logging.py: 85.5%
mssql_python.helpers.py: 89.3%
mssql_python.pooling.py: 90.1%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Suppressed comments (4)

OneBranchPipelines/scripts/build-conda-packages.sh:173

• The verify env name only includes the Python version. On macOS this script is invoked twice on the same agent (native osx-arm64, then cross-target osx-64), so the second conda create -n verify_... can fail with an existing prefix or reuse the wrong-arch env. Include the effective subdir (native vs $CONDA_SUBDIR) in the env name to avoid collisions.

for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="

OneBranchPipelines/stages/build-linux-single-stage.yml:116

• This apt install runs before the later apt-get update in the job, so it can fail on agents with stale apt indices (common on hosted Ubuntu). Add an apt-get update (and ideally noninteractive) before installing qemu-user-static/libc6-arm64-cross.

              # qemu-user-static: run aarch64 ELF binaries on the x86_64 host.
              # libc6-arm64-cross: the aarch64 glibc runtime (loader + libc/libm/...)
              # under /usr/aarch64-linux-gnu so qemu can resolve /lib/ld-linux-aarch64.so.1
              # for the emulated aarch64 conda build/verify (QEMU_LD_PREFIX points here).
              sudo apt-get install -y qemu-user-static libc6-arm64-cross

OneBranchPipelines/steps/conda-publish-step.yml:91

• Installing anaconda-client from PyPI without pinning a version can make releases non-reproducible and may unexpectedly break publishing if a new release introduces behavioral changes. Consider pinning to a vetted version (or using a constraints file) so publish behavior is stable.

        $ErrorActionPreference = 'Stop'
        python -m pip install --upgrade pip
        python -m pip install anaconda-client
        # anaconda-client installs the `anaconda` console script onto PATH.
        anaconda --version

OneBranchPipelines/steps/conda-build-validate-step.yml:126

• The error message references $links (the wheel find-links directory), but conda packages are searched under ${{ parameters.outputDir }}/bld. If this trips, the message will mislead troubleshooting.

      if (-not $built) { Write-Error "No conda packages were produced under $($links)"; exit 1 }

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (2)

OneBranchPipelines/scripts/build-conda-packages.sh:184

• The verify env name is only based on the Python version (e.g. "verify_310"). On macOS this script is invoked twice on the same agent (osx-64 and osx-arm64) sharing the same Miniforge install/outputDir, so the second invocation will fail when it tries to create an env that already exists. Include the target subdir in the env name (and/or delete any existing env before creating it) so the two runs don’t collide.

for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="
  # -c microsoft (ahead of conda-forge) so azure-core/azure-identity/msal resolve from the
  # lean `microsoft` channel, NOT conda-forge whose azure-core recipe over-declares flask/six
  # -> celery/boto3/botocore (~9 MB); see conda-forge/azure-core-feedstock#71.
  # --strict-channel-priority keeps the freshly built local companion + binding authoritative.
  "$conda" create -y -n "$envName" -c "$bld" -c microsoft -c conda-forge --strict-channel-priority --override-channels "python=$py" mssql-python

conda/validate_conda_release.py:67

• read_index_json() uses next(...) to locate the info-*.tar.zst member inside a .conda. If the archive is malformed (missing that member), this will raise StopIteration and produce a stack trace rather than a clear validation failure message. Handle the empty case and raise a ValueError with a helpful message instead.

        with zipfile.ZipFile(path) as zf:
            info_name = next(
                n for n in zf.namelist() if n.startswith("info-") and n.endswith(".tar.zst")
            )
            info_blob = zf.read(info_name)

[Copilot]

🟡 Changes recommended

Three unresolved findings remain, including one critical release-policy validation issue and two moderate validation/promotion issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

conda/validate_conda_release.py:361

• The duplicate check only covers repeated subdir entries. _split() collapses repeated Python values later when validate() builds got_pythons as a set, so --subdir-pythons win-arm64=3.12,3.13,3.13 is accepted even though the release policy says duplicate overrides are rejected. Reject duplicate versions before storing the override.

        result[subdir] = _split(pys)

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Unresolved critical and moderate findings affect workflow token exposure, metadata validation, and Azure DevOps authentication.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

conda/promote_conda_release.py:65

• distribution_from_path coerces the untrusted name, version, and subdir metadata to strings before validating it. A package with version: null (or another non-string value) can therefore become a filename-compatible value such as None and pass the standalone --check-local-only/promotion preflight, even though the release validator rejects non-string index fields. Read these fields with _required_index_string so malformed info/index.json cannot reach the uploader.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

conda/validate_conda_provenance.py:130

• This sends six literal asterisks as the Authorization header instead of the validated SYSTEM_ACCESSTOKEN, so every real Azure DevOps request will be unauthenticated and both provenance and publication-lock checks will fail (or, if an endpoint is anonymously readable, lose the intended identity check). Send the token as the Azure DevOps bearer credential; update the synthetic HTTP test to assert the real header without printing it.

        headers={"Authorization": f"Bearer {token}"},

tests/test_036_conda_provenance.py:261

• The regression test currently expects the placeholder ******, so it would pass while the production client discards the access token. Assert the bearer form built from the fixture token so this test catches unauthenticated Azure DevOps calls.

    assert request.get_header("Authorization") == "Bearer synthetic-token"

• Files reviewed: 13/13 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

A moderate metadata-validation issue remains, and production publication qualification is not established.

Review details

Suppressed comments (1)

conda/promote_conda_release.py:65

• These fields come from authoritative info/index.json, but coercing them with str() lets malformed metadata through the standalone preflight: for example, version: true can become "True" in a canonically named archive, and a numeric subdir can match its parent directory. That makes --check-local-only (and a direct promotion invocation) accept metadata that the release validator rejects. Use the same strict non-empty trimmed-string validation here so the publisher cannot derive identities from non-string metadata.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

• Files reviewed: 13/13 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Two moderate issues remain in metadata preflight validation and Azure DevOps authorization handling.

Review details

Suppressed comments (2)

conda/promote_conda_release.py:65

• This publisher preflight normalizes metadata instead of enforcing the release metadata contract. For example, a canonical archive whose version is " 1.13.0 " is accepted here after stripping, even though validate_conda_release requires a non-empty trimmed string; invoking this local preflight independently can therefore stage metadata that the authoritative gate rejects. Read name, version, and subdir with _required_index_string just as build is read below.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

conda/validate_conda_provenance.py:130

• The request sends the literal value ****** as the Authorization header, so the validated SYSTEM_ACCESSTOKEN is never transmitted. Every Azure DevOps GET will be unauthenticated and both provenance and publication-lock guards will fail in a real pipeline; send the token as a Bearer credential and keep redaction limited to logs/assertions.

        headers={"Authorization": f"Bearer {token}"},

• Files reviewed: 13/13 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Two moderate findings require fixes in the publisher interpreter invocation and promotion metadata validation.

Review details

Suppressed comments (2)

OneBranchPipelines/steps/conda-publish-step.yml:133

• $(Join-Path ...) is interpreted by Azure Pipelines as a $(variable) macro before PowerShell evaluates this string, so the logging command does not reliably prepend the venv's Scripts directory. The following task therefore resolves the agent's python instead of the hash-locked publisher interpreter; after the intentional disable guard is removed, publication can run with the wrong environment or fail to import binstar_client. Use PowerShell concatenation/parenthesized invocation that does not contain Azure macro syntax.

        Write-Host "##vso[task.prependpath]$(Join-Path $publisherEnvironment 'Scripts')"

conda/promote_conda_release.py:76

• The credential-boundary preflight coerces name, version, and subdir to strings and strips whitespace, unlike the strict _required_index_string checks used by the release validator. A malformed archive with metadata such as a numeric version or a whitespace-padded version can therefore pass --check-local-only (and recovery) when its filename matches the normalized value. Validate these three fields with _required_index_string before using them to derive the canonical upload identity.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Critical publication paths remain reachable, and additional validation inconsistencies must be fixed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

OneBranchPipelines/steps/conda-release-step.yml:190

• The release-readiness/validate-only path only invokes validate_conda_release.py, whose collector never compares each archive basename with name-version-build.ext. Consequently a renamed archive can pass the readiness gate and appear in the dry-run plan, while the credentialed publish job later rejects it in promote_conda_release.py. Enforce the same canonical-basename preflight in this gate (or move the shared check into the validator) so validate-only and publish readiness validate the same exact artifacts.

        python "$gate" @gateArgs

conda/promote_conda_release.py:69

• These fields are normalized with str(...).strip() instead of validated. An archive whose info/index.json contains name: "mssql-python ", version: "1.13.0 ", or subdir: "win-64 " can therefore pass this standalone promotion/cleanup CLI and its canonical-name/folder checks, even though validate_conda_release.collect_packages() rejects the same malformed metadata via _required_index_string. Use the strict helper for all three fields so malformed or non-string metadata fails closed before any client operation.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

• Files reviewed: 11/11 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Six unresolved findings remain, including one critical dependency-integrity issue and five moderate validation gaps.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (5)

OneBranchPipelines/conda-release-pipeline.yml:235

• This dry-run enumerates files and prints them as the exact upload plan, but it never applies the publisher's distribution_from_path preflight. The readiness gate only validates info/index.json and the matrix, so a valid archive renamed away from its metadata-derived canonical basename will produce a successful dry-run and then be rejected when publishToConda=true; reuse the same local preflight (or duplicate its canonical-name validation) before printing the plan.

                      foreach ($p in ($pkgs | Sort-Object FullName)) {
                        $subdir = Split-Path -Leaf (Split-Path -Parent $p.FullName)
                        $sha256 = (Get-FileHash -LiteralPath $p.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
                        Write-Host "  SHA256: $sha256  $subdir/$($p.Name)"
                        Write-Host "  WOULD RUN: anaconda upload --user $env:CONDA_CHANNEL --label $env:CONDA_LABEL --skip-existing  $subdir/$($p.Name)"

conda/promote_conda_release.py:68

• These fields are coerced with str(...).strip(), so a malformed info/index.json can still pass this standalone publisher preflight: for example, a numeric version or a whitespace-padded version becomes the expected canonical string and can be accepted for upload. The release gate is stricter, but this helper is also the public local/credential-boundary validator; use _required_index_string for name, version, and subdir just as this code already does for build.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

conda/validate_conda_release.py:327

• collect_packages() validates the embedded fields and parent folder but never compares the archive basename with the metadata-derived <name>-<version>-<build>.<ext> name. A renamed archive therefore passes ValidateConda and the non-publishing dry run, then fails only in promote_conda_release.py's later local preflight before upload. Enforce this canonical-name check in the readiness gate as well (ideally through a shared helper) so a successful dry run represents a publishable set.

        index = read_index_json(path)
        name = _required_index_string(index, "name", path)
        version = _required_index_string(index, "version", path)
        subdir = _required_index_string(index, "subdir", path)
        build = _required_index_string(index, "build", path)

conda/validate_conda_release.py:148

• search() only inspects the first pyXY token. A malformed build such as py312_py313_0 with python 3.12.* is therefore accepted as 3.12 even though this function documents that build tokens must agree, allowing inconsistent metadata through the matrix gate. Collect every build-token match so the existing conflict check rejects this case, and add a regression test.

    match = _PY_TAG_RE.search(str(index.get("build", "")))
    if match:
        minors.add(f"{match.group(1)}.{match.group(2)}")

conda/validate_conda_release.py:294

• The matrix key here is reduced to (subdir, python), so two mssql-python archives for the same platform/Python with different build values collapse into one entry and pass validation. A 29-package tree can therefore be reported release-ready and both builds will be uploaded, despite the release contract being one archive per matrix cell; reject multiple distinct builds per (name, version, subdir, python) (or enforce the exact expected package set).

        sub_expected = subdir_pythons.get(sub, expected_pythons)
        got_pythons = sorted({p["python"] for p in bindings if p["python"]})
        missing = [py for py in sub_expected if py not in got_pythons]

• Files reviewed: 11/11 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Three moderate issues remain in dry-run parity, publisher metadata validation, and required-subdirectory parsing.

Review details

Suppressed comments (3)

OneBranchPipelines/conda-release-pipeline.yml:201

• The dry-run only hashes and prints every discovered filename; it does not run the publisher's canonical-basename preflight. A renamed .tar.bz2 can pass the metadata gate and produce a successful-looking upload plan here, but conda-publish-step.yml will reject the same artifact in promote_conda_release.py --check-local-only before upload. Run the same local preflight (or equivalent canonical checks) in the dry-run so its result is actionable.

                      $pkgs = @(Get-ChildItem -Path $root -Recurse -Include *.conda, *.tar.bz2)
                      if ($pkgs.Count -eq 0) { Write-Error "No conda packages found under $root."; exit 1 }

conda/promote_conda_release.py:68

• This publisher-side preflight coerces and strips name, version, and subdir instead of enforcing the Conda metadata schema. For example, an archive whose info/index.json has version: " 1.15.0 " can pass --check-local-only (and cleanup/promotion when invoked directly) because the stripped value matches the expected version, even though the release validator rejects it. Use the same non-empty, trimmed-string validation here so this standalone publication/recovery boundary cannot accept malformed metadata.

    package = str(index.get("name", "")).strip()
    version = str(index.get("version", "")).strip()
    subdir = str(index.get("subdir", "")).strip()

conda/validate_conda_release.py:343

• Filtering empty tokens silently turns a malformed policy such as --required-subdirs win-64,,linux-64 into a weaker two-entry policy. An operator typo can therefore omit required platforms and let an incomplete package set pass; reject empty elements (while preserving an entirely blank value as an empty policy that validate() rejects) instead of dropping them.

    return [x.strip() for x in value.split(",") if x.strip()]

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The publish dependency lock is incomplete, and cleanup can delete a public label without staging-label restrictions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

The pipeline has a stale version default and an incorrect dry-run upload path.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

OneBranchPipelines/conda-release-pipeline.yml:53

• This default will become stale on the next version bump: the repository's release procedure updates only mssql_python/__init__.py, setup.py, and PyPI_Description.md, so normal release PRs will not update this pipeline parameter. A stale default makes the default validation/publish invocation reject the selected artifacts unless an operator remembers an override; leave the parameter empty and require it only for production, or derive it from the selected artifact.

OneBranchPipelines/conda-release-pipeline.yml:235

• The dry-run plan prints $subdir/$($p.Name) as the upload argument, but the downloaded archive is actually at $p.FullName under conda-artifacts/conda and the job never changes to a directory containing that relative path. This makes the advertised anaconda upload plan non-executable and can mislead operators reviewing a release; print the same local path that the publishing step passes to Anaconda (and quote it if needed).

                        Write-Host "  WOULD RUN: anaconda upload --user $env:CONDA_CHANNEL --label $env:CONDA_LABEL --skip-existing  $subdir/$($p.Name)"

• Files reviewed: 12/12 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Critical validation and moderate dry-run-label findings remain unresolved in high-risk release automation.

Review details

Suppressed comments (1)

OneBranchPipelines/conda-release-pipeline.yml:235

• The dry-run prints --label $env:CONDA_LABEL, but the real publisher uploads each archive to the generated $stagingLabel (conda-publish-step.yml:227-230) and only then promotes that label. This makes the promised exact plan misleading: an operator cannot review the actual staging-label upload and promotion sequence from this output. Print the generated staging label and the subsequent promotion/cleanup operations, or explicitly label this as only a final-label summary.

                        Write-Host "  WOULD RUN: anaconda upload --user $env:CONDA_CHANNEL --label $env:CONDA_LABEL --skip-existing  $subdir/$($p.Name)"

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

The release and publishing automation is high-impact, and the current-source ADO dry run has not been executed.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 0 new
• Review effort level: Lite