Make tests work on hardened runner - #4875
Merged
Merged
Conversation
Co-authored-by: Isaac
Co-authored-by: Isaac
The hardened runner blocks direct access to public registries. Use JFrog Artifactory as a proxy with OIDC authentication. Co-authored-by: Isaac
The jf goc command only works with jf go, not native go commands. Set GOPROXY and .netrc for native go toolchain. Co-authored-by: Isaac
Replace jf config show (no JSON format flag) with direct OIDC token exchange, matching the pattern from the hardened runner docs for uv/cargo. Co-authored-by: Isaac
Use jf goc + jf go mod download to pre-populate the module cache via JFrog. Native go commands then work from cache. Co-authored-by: Isaac
Co-authored-by: Isaac
Collaborator
|
Commit: 5270413
19 interesting tests: 10 SKIP, 7 KNOWN, 2 flaky
Top 21 slowest tests (at least 2 minutes):
|
Co-authored-by: Isaac
The hardened runner doesn't have Node.js pre-installed, which is needed by the run-local-node acceptance test. Co-authored-by: Isaac
The test requires npm registry access which is blocked on the hardened runner. Disable until npm is routed through JFrog. Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 08:35 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 08:36 — with
GitHub Actions
Failure
GitHub Actions
Failure
The existing runners are already hardened, so no need to switch to the hardened-optin group. Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 08:40 — with
GitHub Actions
Failure
GitHub Actions
Failure
Windows runners are also hardened and can't reach proxy.golang.org. Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 08:47 — with
GitHub Actions
Failure
GitHub Actions
Failure
denik
approved these changes
Apr 1, 2026
PowerShell splits -modfile=tools/go.mod into separate tokens. Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 08:51 — with
GitHub Actions
Failure
GitHub Actions
Failure
This ensures all test jobs that use the shared action get the JFrog Go module proxy configured on hardened runners. Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 09:14 — with
GitHub Actions
Failure
GitHub Actions
Failure
…test jobs to custom runners Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 09:20 — with
GitHub Actions
Failure
GitHub Actions
Failure
Co-authored-by: Isaac
pietern
had a problem deploying
to
test-trigger-is
April 1, 2026 09:28 — with
GitHub Actions
Failure
GitHub Actions
Failure
andrewnester
approved these changes
Apr 1, 2026
pietern
marked this pull request as ready for review
April 1, 2026 09:40
pietern
requested review from
anton-107,
shreyas-goenka and
simonfaltum
as code owners
April 1, 2026 09:40
Contributor
Suggested reviewersBased on git history of the changed files, these people are best suited to review:
Confidence: high Eligible reviewersBased on CODEOWNERS, these people or teams could also review: @andrewnester, @anton-107, @shreyas-goenka, @simonfaltum Suggestions based on git history of 6 changed files (5 scored). See CODEOWNERS for path-specific ownership rules. |
denik
reviewed
Apr 1, 2026
| @@ -1,5 +1,6 @@ | |||
| Badness = "need to enable NPM registry access" | |||
Contributor
There was a problem hiding this comment.
We should also consider vendoring dependencies here like we did with Python.
hectorcast-db
pushed a commit
to databricks/databricks-sdk-py
that referenced
this pull request
Apr 7, 2026
## Summary Hardened GitHub Actions runners block direct access to public registries (PyPI, files.pythonhosted.org). This adds OIDC-based JFrog Artifactory authentication so CI can download Python packages through the `db-pypi` proxy. Uses the same `jfrog/setup-jfrog-cli` action as [CLI PR #4875](databricks/cli#4875). ## What changed - **New composite action** (`.github/actions/setup-jfrog-pypi/action.yml`): Uses `jfrog/setup-jfrog-cli` for OIDC authentication (same action + SHA as the CLI repo), then sets `UV_INDEX_URL` using the token from the action's `oidc-token`/`oidc-user` outputs. - **`test.yml`**: Added `id-token: write` permissions, JFrog setup step, and `uv lock` before `make dev test`. - **`push.yml`**: Added workflow-level `id-token: write` permissions (needed for `workflow_call` to `test.yml`). Added JFrog setup + `uv lock` to `fmt` and `check-manifest` jobs. - **`Makefile`**: Added `fix-lockfile` target. All existing targets are unchanged. ## How it works `uv.lock` stores full registry URLs per package. Setting `UV_INDEX_URL` to JFrog causes `uv sync --locked` to fail (stale lockfile). To work around this, CI runs `uv lock` after JFrog setup, which rewrites the lockfile with JFrog URLs while keeping the same versions. Then `make dev test` / `make dev fmt` work unchanged with `--locked`. The lockfile rewrite is ephemeral (never committed). In the `fmt` job, `git checkout -- '*.lock'` restores the committed lockfile after formatting, then `make fix-lockfile` normalizes any accidentally committed proxy URLs. `git diff --exit-code` catches both unformatted code and dirty lockfiles. ## `make fix-lockfile` Replaces JFrog proxy URLs with their public PyPI equivalents in all `*.lock` files. Prevents proxy URLs from being accidentally committed to the repo. **When to use it**: If you run `uv lock` while `UV_INDEX_URL` points to JFrog (e.g. after sourcing CI env vars), run `make fix-lockfile` before committing to normalize URLs back to public PyPI. **CI enforcement**: The `fmt` job runs this before `git diff --exit-code`, so committed proxy URLs fail CI. ## Out of scope - **`tagging.yml`**: Generated from the Universe codegen template (`openapi/genkit/sync/workflows/tagging.yml`). JFrog setup needs to be upstreamed there. - **`release.yml` / `release-test.yml`**: Need special publish runners per the migration guide. Separate follow-up with the security team. - **Runner migration**: Switching to hardened runner labels can be done as a follow-up. - **`Trigger Tests` CI failure**: Pre-existing IP allowlist issue (DECO-26816). NO_CHANGELOG=true
github-merge-queue Bot
pushed a commit
to databricks/databricks-sdk-java
that referenced
this pull request
Apr 8, 2026
## Summary Route Maven dependency resolution through JFrog Artifactory on hardened runners that block direct access to Maven Central. Authenticate via GitHub Actions OIDC (zero stored secrets). Modeled after the CLI ([#4875](databricks/cli#4875)), Go SDK ([#1609](databricks/databricks-sdk-go#1609)), and Python SDK ([#1379](databricks/databricks-sdk-py#1379)). ## What changed - **New composite action** (`.github/actions/setup-build-environment/action.yml`): Sets up JFrog CLI via OIDC, installs the JDK, and generates a `~/.m2/settings.xml` that mirrors all Maven repositories through JFrog (`db-maven`). Skipped on macOS (not hardened). - **`push.yml`**: Added workflow-level `id-token: write` permission for OIDC. Switched `fmt`, `unit-tests` (Linux), and `check-lock` jobs to `databricks-protected-runner-group`. macOS tests remain on public runners. Used matrix `include` to map OS strings to runner configs, preserving the original check names for branch protection compatibility. - **`Makefile`**: Added `fix-lockfile` target. - **`lockfile.json`**: Regenerated to pick up `commons-configuration2` 2.13.0 (was out of date since PR #743 bumped the POM without regenerating the lockfile). ## Why settings.xml is needed Hardened runners terminate SSL handshakes to `repo.maven.apache.org`. Unlike Go (`GOPROXY` env var) or Python (`UV_INDEX_URL`), Maven has no environment variable to redirect repository access. The standard mechanism is a `~/.m2/settings.xml` with a `<mirror>` entry that routes all repository requests through JFrog. ## The lockfile problem Maven's `lockfile.json` stores the full download URL in each dependency's `resolved` field. When Maven resolves through JFrog, these become `https://databricks.jfrog.io/artifactory/db-maven/...` instead of `https://repo.maven.apache.org/maven2/...`. If someone runs `make lock` while Maven is routed through JFrog (CI, VPN), the lockfile would contain proxy URLs that shouldn't be committed. ### make fix-lockfile Replaces JFrog proxy URLs with their public Maven Central equivalents in all `lockfile.json` files. Prevents proxy URLs from being accidentally committed. **When to use it**: Run `make fix-lockfile` before committing if you regenerated the lockfile while Maven was routing through JFrog. **CI enforcement**: The `fmt` job runs `make fix-lockfile` followed by `git diff --exit-code`, so any committed proxy URLs will fail CI. ## Cache ordering The Maven cache (`~/.m2/repository`) must be restored **before** the composite action runs. The composite action generates `~/.m2/settings.xml` with a fresh OIDC token. If the cache step runs after and restores `~/.m2` (including a stale `settings.xml` from a previous run), the expired token causes 401 errors. Caching only `~/.m2/repository` (not all of `~/.m2`) prevents this. ## Out of scope - **release.yml**: Publishes to Maven Central via Sonatype. Needs special publish runners per the migration guide. Separate follow-up with the security team. - **tagging.yml**: Generated from Universe (openapi/genkit/sync/workflows/tagging.yml). JFrog setup needs to be upstreamed to the genkit template. - **conftest.yml**: Only downloads conftest binary from GitHub Releases (not Maven). Can stay on ubuntu-latest. ## Test plan - [x] Verify mvn --errors spotless:check passes (fmt job) - [x] Verify mvn --errors test passes on Linux (Java 8, 11, 17, 20) - [x] Verify mvn --errors test passes on macOS (Java 8, 11, 17, 20) - [x] Verify make check-lock passes (lockfile validation) - [x] Verify conftest passes on the new workflow files - [x] Verify check names match branch protection rules NO_CHANGELOG=true --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
denik
pushed a commit
that referenced
this pull request
May 20, 2026
## Summary - Configure JFrog Artifactory as Go module proxy via OIDC for Linux and Windows runners - Disable `run-local-node` acceptance test (needs npm registry access via JFrog) - Add setup-build-environment to testmask triggers so CI changes test all targets - Switch secondary test jobs (aitools, ssh, pipelines) to use custom runner groups ## Test plan - [x] Verify `make test` passes on Linux, Windows, macOS - [x] Verify `test-exp-aitools`, `test-exp-ssh`, `test-pipelines` are triggered and pass This pull request was AI-assisted by Isaac.
radakam
added a commit
that referenced
this pull request
Aug 6, 2026
The test has been disabled since #4875, which took NPM registry access away from the runners. It needed the registry only for the app fixture: express, and the --prepare-environment run that installs it. The fixture is now a standard-library HTTP server started directly with node, so nothing is installed at test time. /shutdown answers with Connection: close so server.close() can complete and the app exits on its own, which lets wait $PID assert that the CLI exited zero. Ports come from free_port.py, a failed run reaps the app and the CLI by port, and the Timeout overrides go away because the run takes well under a second rather than 2m. --debug stays, unlike in the python sibling where it needs debugpy from PyPI: node debugging is only NODE_OPTIONS=--inspect. The test asserts both the port the CLI reports and that node listens on it, so --debug-port keeps its coverage. npm is out of the fixture too. The rejecting proxy catches npm run contacting registry.npmjs.org for its update notifier, which npm skips when CI is set, so keeping npm would have failed locally only and needed a config workaround.
radakam
added a commit
that referenced
this pull request
Aug 6, 2026
The test has been disabled since #4875, which took NPM registry access away from the runners. It needed the registry only for the app fixture: express, and the --prepare-environment run that installs it. The fixture is now a standard-library HTTP server started directly with node, so nothing is installed at test time. /shutdown answers with Connection: close so server.close() can complete and the app exits on its own, which lets wait $PID assert that the CLI exited zero. The app throws when PORT is unset rather than letting listen() pick a random port and failing fifteen seconds later as a proxy timeout; PORT is read in preference to DATABRICKS_APP_PORT because it is what Node apps read and what GetBaseEnvVars sets for that reason, so it covers an alias the python fixture cannot. Ports come from free_port.py, and the Timeout overrides go away because the run takes a few seconds rather than 2m. --prepare-environment comes back with it. The flag had no test left anywhere: the python sibling dropped it for needing PyPI, and neither PrepareEnvironment has a unit test. npm install only reaches the registry when there is something to install, so the fixture keeps an empty dependency list and gains a build script, and a single run covers both halves of NodeApp.PrepareEnvironment: the install has to succeed for the build script to run at all. The build script is a file rather than an inline command because npm's shell quotes inline arguments differently on Windows, and the assertion greps for the exact output. With nothing to install the one request npm still makes on its own is the check for a newer npm, which the rejecting proxy blocks, so app/.npmrc turns that check off. --debug stays too, unlike in the python sibling where it needs debugpy from PyPI: node debugging is only NODE_OPTIONS=--inspect. The test asserts both the port the CLI reports and that node listens on it, so --debug-port keeps its coverage. The CLI prints its line only after the proxy accepts connections, so the curl that reaches the app can beat it and that grep is retried. A failed run reaps the app, proxy and debug ports. node binds the inspector port before it runs app.js, so the debug port is the only handle on an app that died before it listened.
asnare
pushed a commit
to asnare/databricks-cli
that referenced
this pull request
Aug 8, 2026
## Changes Re-enable `acceptance/cmd/workspace/apps/run-local-node`, disabled since databricks#4875 for needing NPM registry access. The fixture is now a `node:http` server run directly with `node`, so `express` and the 2m/10m timeouts are gone. Ports come from `free_port.py`, `/shutdown` lets the app exit on its own so `wait $PID` asserts the CLI's exit code, and a failing run reaps the app, proxy and debug ports. `--prepare-environment` and `--debug` stay. The app has no dependencies, so `npm install` fetches nothing, and `app/.npmrc` turns off npm's check for a newer npm, which is the one request it still makes on its own; one run covers both halves of `NodeApp.PrepareEnvironment` and the `--entry-point` override. The build script is a file rather than an inline command because npm quotes inline arguments differently on Windows and the assertion greps for its exact output. Node debugging is only `NODE_OPTIONS=--inspect`, so the test asserts the port the CLI reports and that node listens on it. ## Why Only the fixture needed the registry, for `express`. `--prepare-environment` is untested anywhere else: the python sibling dropped it for needing PyPI, and neither `PrepareEnvironment` has a unit test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run-local-nodeacceptance test (needs npm registry access via JFrog)Test plan
make testpasses on Linux, Windows, macOStest-exp-aitools,test-exp-ssh,test-pipelinesare triggered and passThis pull request was AI-assisted by Isaac.