Add support for UC secrets by andrewnester · Pull Request #5861 · databricks/cli

andrewnester · 2026-07-08T12:48:12Z

Changes

Add support for Unity Catalog (UC) secrets as a first-class DAB resource type under the direct deployment engine.

Notes:

Secret values are never persisted to deployment state — the remote plaintext is not returned by the API and is intentionally omitted from state to avoid leaking secrets
New validation mutator (ValidateSecretValueIsVariable) that rejects plain-text value fields at bundle validation time — the value must be a variable reference (e.g. ${var.my_secret}) so secrets are never committed to databricks.yml

Also requires #5896

Example

bundle:
  name: my-bundle

variables:
  my_secret_value:
    description: "Secret value passed at deploy time"

resources:
  secrets:
    my_secret:
      catalog_name: main
      schema_name: default
      name: my_api_key
      value: ${var.my_secret_value}
      comment: "API key for external service"

Tests

Acceptance tests covering create/update/delete, update-value, direct-only enforcement, and the plain-text validation error

eng-dev-ecosystem-bot · 2026-07-08T13:19:46Z

Integration test report

Commit: bb8e450

Run: 31090074644

	Env	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux	4	4	293	1109	5:22
💚	aws windows	4	4	295	1107	6:31
💚	azure linux	4	4	292	1109	6:02
💚	azure windows	4	4	294	1107	5:05
💚	gcp linux	1	5	293	1109	6:09
💚	gcp windows	1	5	295	1107	5:25

8 interesting tests: 4 RECOVERED, 4 SKIP

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R
💚	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
5:02	aws windows	TestAccept
4:25	gcp windows	TestAccept
3:03	azure windows	TestAccept

shreyas-goenka · 2026-07-13T08:19:36Z

+variables:
+  secret_value:
+    description: "Test secret value"
+    default: "initial-secret-value"


Should we disallow this pattern? Error if we detect this? Secrets should not be stored in plain text in YAML.

janniklasrose · 2026-07-14T15:21:54Z

+// Terraform provider implementation:
+// https://github.com/databricks/terraform-provider-databricks/blob/main/catalog/resource_secret.go


I think we can drop this comment

janniklasrose · 2026-08-05T11:37:21Z

 		v.SchemaName = resolveSchema(b, v.CatalogName, v.SchemaName)
 		v.CatalogName = resolveCatalog(b, v.CatalogName)
 	}
+	for _, s := range b.Config.Resources.Secrets {


Curious (about capture_uc_dependencies in general): is this just a quality of life improvement if customers have their catalog & schema in DABs but don't use the in-bundle reference?

yes, correct

janniklasrose · 2026-08-05T11:45:48Z

+		return nil, err
+	}
+
+	// SDK does not support include_value in the GetSecretRequest, so we use the API directly.


any reason why? We can (ask to) fix this upstream and switch to the SDK method later

Not sure why, it's a query param though so I suspect there might not be support for such in SDK yet

janniklasrose · 2026-08-05T11:47:50Z

+	state.Value = state.SecretValue
+	response, err := r.client.SecretsUc.CreateSecret(ctx, catalog.CreateSecretRequest{
+		Secret: state.Secret,
+	})
+	// Clear the plaintext so it is not written to the state file.
+	// Fingerprint already captures whether the value changed.
+	state.Value = ""


why are we writing state.SecretValue into state.Value, not using it, then clearing state.Value again? is the read of state.Secret derived from Value? if so a comment would help clarify that side-effect.

ditto below on DoUpdate

why are we writing state.SecretValue into state.Value, not using it,

We use it below when passing state.Secret, state.Value is practically state.Secret.Value

a comment like

// temporarily assign `state.Value` so `state.Secret` has access

would help the (future) reader

## Release v1.12.0 ### CLI * `databricks aitools install` now supports Gemini CLI, installing Databricks agent skills into its skills directory. * `databricks aitools install` now supports Pi, installing Databricks agent skills into its skills directory. * A locally built CLI (`go build`, without release flags) now reports the next release version with a `-dev` prerelease, e.g. `1.12.0-dev+abcdef123456`, instead of `0.0.0-dev+abcdef123456`. The old string sorted below every published release even though a local build is newer than the latest release; the new one sorts above the latest release and below the release it will become, matching what goreleaser already produces for snapshot builds. * Added the `databricks environments setup-local` command, which provisions (or updates) a local Python environment matched to a Databricks compute target. It resolves the target to an environment key, fetches the pinned Python version, databricks-connect version, and dependency constraints published for that key, then provisions a matched `.venv` with uv. ### Bundles * Added a `cascade_on_destroy` field to the pipeline resource to control whether destroying a pipeline also deletes its datasets (MVs, STs, Views). When unset, the server default applies; set `cascade_on_destroy: false` to retain the datasets on destroy. Supported with the direct deployment engine ([#5846](#5846)). * Fix `bundle.deployment.lock.force` being ignored. The `--force-lock` flag's default value overwrote the value configured in `databricks.yml`, so setting the field had no effect and a stale deployment lock could only be overridden with the flag. ([#6188](#6188)) * direct: experimental `job_runs` now sends a CLI-managed idempotency token on every run-now, so an SDK retry after a lost response returns the same run. Configured `idempotency_token` values are rejected. * direct: the experimental `job_runs` resource now waits for the triggered run to finish, so other resources can reference its outcome (e.g. `${resources.job_runs.nightly.state.result_state}`). A run that does not succeed fails the deploy, naming the failed task, and is run again on the next deploy. If a deploy is interrupted while waiting, the next one resumes waiting on the same run. * direct: Fixed model serving `telemetry_config` drift and applied planned telemetry updates. Unsupported endpoint types now fail when telemetry is applied; create may still succeed because it drops the field ([#6106](#6106)). * The `cli_version` field in the direct engine's deployment state (`resources.json`) now records the CLI version that last wrote the state. Previously it kept the version of the CLI that first created the state. * Add support for UC secrets resource ([#5861](#5861)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.166.0 to v0.169.0. * Bump Terraform provider from v1.124.0 to v1.126.0 ([#6250](#6250)).

andrewnester temporarily deployed to test-trigger-is July 8, 2026 12:48 — with GitHub Actions Inactive

andrewnester requested review from denik, janniklasrose, pietern and shreyas-goenka July 8, 2026 12:49

andrewnester temporarily deployed to test-trigger-is July 9, 2026 08:56 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 09:28 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 09:51 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 10:05 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 10:51 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 11:21 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 11:45 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 12:45 — with GitHub Actions Inactive

This was referenced Jul 9, 2026

Add "Secrets" to DAB Resources #3689

Closed

internal: Add support for 'sensitive' JSON tag for bundle config fields #5896

Merged

shreyas-goenka reviewed Jul 13, 2026

View reviewed changes

andrewnester requested a review from shreyas-goenka July 14, 2026 09:51

andrewnester temporarily deployed to test-trigger-is July 14, 2026 09:52 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 14, 2026 14:57 — with GitHub Actions Inactive

andrewnester added 3 commits July 30, 2026 13:15

regenrate

4ccd878

next changelog

25a78fd

fixes

77d44ee

denik reviewed Jul 30, 2026

View reviewed changes

Comment thread acceptance/bundle/refschema/out.fields.txt Outdated

Comment thread bundle/direct/dresources/resources.yml Outdated

Comment thread bundle/direct/dresources/secret.go Outdated

andrewnester added 2 commits August 3, 2026 15:50

compare on value

c904d13

diff on value

65899b5

andrewnester requested a review from denik August 3, 2026 13:58

andrewnester and others added 4 commits August 3, 2026 15:59

remove unused code

a4c3eff

test fixes

8c52c03

refschema + remove not drift

8d772f9

Merge branch 'main' into feat/uc-secrets

ac7171a

janniklasrose reviewed Aug 5, 2026

View reviewed changes

andrewnester requested a review from janniklasrose August 5, 2026 12:18

Merge branch 'main' into feat/uc-secrets

08ad2fc

janniklasrose approved these changes Aug 5, 2026

View reviewed changes

andrewnester and others added 3 commits August 5, 2026 15:59

Merge branch 'main' into feat/uc-secrets

a4d1b10

update out files

9876d35

update no drift

c3ea248

denik reviewed Aug 5, 2026

View reviewed changes

Comment thread ...ptance/bundle/resources/postgres_endpoints/update_autoscaling/out.plan.no_change.direct.json

use the same strcut as state

ce49551

andrewnester requested a review from denik August 5, 2026 15:50

andrewnester and others added 4 commits August 5, 2026 17:54

fixes

3d98f02

Merge branch 'main' into feat/uc-secrets

970fdcc

fix structs

1235f60

fix invariant

bb8e450

andrewnester disabled auto-merge August 6, 2026 11:01

andrewnester merged commit 7eb1fc2 into main Aug 6, 2026
48 checks passed

andrewnester deleted the feat/uc-secrets branch August 6, 2026 11:02

rugpanov mentioned this pull request Aug 12, 2026

Update Databricks CLI to v1.12.1 databricks/databricks-vscode#2109

Merged

		// Terraform provider implementation:
		// https://github.com/databricks/terraform-provider-databricks/blob/main/catalog/resource_secret.go

Uh oh!

Conversation

andrewnester commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example

Tests

Uh oh!

eng-dev-ecosystem-bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andrewnester commented Jul 8, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jul 8, 2026 •

edited

Loading