Repository navigation
Description
Describe the issue
bundle deployment bind fails on the direct engine when experimental.immutable_folder: true is set. Bind calculates a plan on a configuration that does not hold resources.internal_immutable_snapshots.immutable. So every resource that references ${workspace.file_path} cannot resolve, and the plan fails. This blocks moving an existing resource into a bundle with bind.
Configuration
bundle:
name: example-bundle
engine: direct
experimental:
immutable_folder: true
resources:
jobs:
example_job:
name: example-job
tasks:
- task_key: main
environment_key: default
spark_python_task:
python_file: ./main.py
parameters:
- --input
- ${workspace.file_path}/input.json
environments:
- environment_key: default
spec:
environment_version: "5"
schemas:
example_schema:
catalog_name: main
name: example_schemaSteps to reproduce the behavior
- Create a job and a schema in the workspace, outside the bundle.
- Use a target that has never deployed, so the deployment state is empty.
- Run
databricks bundle deployment bind example_job <job id> --auto-approve. - Run
databricks bundle deploy --select jobs.example_job. This writes the snapshot resource into the state. Then run step 3 again.
Expected Behavior
Bind writes the job ID into the state. The next bundle plan shows the job as update, with its paths in the snapshot folder.
Actual Behavior
Step 3 fails for every resource key, a schema key included:
Error: invalid dependency "${resources.internal_immutable_snapshots.immutable.full_path}", no such node "resources.internal_immutable_snapshots.immutable"
After step 4, bind plans the snapshot as delete and fails:
Error: cannot plan resources.jobs.example_job: cannot resolve "${resources.internal_immutable_snapshots.immutable.full_path}": internal error: resources.internal_immutable_snapshots.immutable: action is "delete" missing new_state
Error: planning failed
Workaround: set immutable_folder: false for the bind only, then set it back to true. The next plan shows skip for a bound schema and update for a bound job, with snapshot paths.
A variable reference in experimental.immutable_folder is not a workaround. With ${var.immutable_folder}, the string parameter ${workspace.file_path}/input.json keeps the plain files/ path, but python_file moves to the snapshot path.
OS and CLI version
macOS, Databricks CLI v1.21.0. The bind flow in bundle/direct/bind.go is the same at v1.22.1, and v1.23.0 does not change that file.
Is this a regression?
Unknown.
Detailed plan
Not attached. Bind fails before it prints a plan.
Debug Logs
Not attached.
Activity