Improve schedule YAML missing-file errors by surfacing a direct validation failure

[Copilot]

When create_job points to a non-existent local YAML (for example, a wrong pipeline.yml path), schedule loading currently emits a deeply nested multi-schema validation blob instead of a direct actionable error. This change makes missing file references fail early with a clear message.

• Error normalization at file-ref source

  • Updated FileRefField._deserialize to catch FileNotFoundError/OSError from load_file(...) and raise a marshmallow ValidationError with a consistent "No such file or directory: <path>" message.

• Union fallback short-circuit for missing local files

  • Updated UnionField._deserialize to stop aggregating alternative schema errors once a file-not-found validation message is present, preventing noisy cascaded errors from unrelated union branches.

• Schedule regression coverage

  • Added a schedule unit test for a non-existent pipeline file reference to assert:
    • file-not-found text is present,
    • it appears once (no repeated nested cascade),
    • unrelated fallback text is not surfaced.

# FileRefField._deserialize
try:
    data = load_file(path)
except (FileNotFoundError, OSError) as e:
    raise ValidationError(f"{NO_SUCH_FILE_OR_DIRECTORY_ERROR}: {path}") from e

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• centralus-2.in.applicationinsights.azure.com
  • Triggering command: /home/REDACTED/bin/azsdk /home/REDACTED/bin/azsdk ingest-telemetry --client-type copilot-cli --event-type skill_invocation --session-id 366c4e6f-aa9c-4246-b4e2-db383393cc01 --skill-name find-package-skill (dns block)
• pypi.org
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python python -m pip install pytest-mock (dns block)
• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Problem

When a user creates a schedule with a wrong or non-existent pipeline.yml path, the SDK throws a confusing, deeply nested JSON validation error like:

{
  "create_job": [
    { "_schema": ["In order to specify an existing jobs, please provide either of the following prefixed with 'azureml:'..."] },
    { "_schema": ["Not supporting non file for create_job"] },
    { "job": [{ "_schema": ["..."] }, ["No such file or directory: CLI\\Pipeline\\..\\pipeline.yml"]] },
    ...
  ]
}

This is tracked in ICM incident 594180941.

Root Cause

In sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py, the FileRefField._deserialize method calls load_file(path) without catching FileNotFoundError:

def _deserialize(self, value, attr, data, **kwargs):
    if isinstance(value, str) and not value.startswith(FILE_PREFIX):
        base_path = Path(self.context[BASE_PATH_CONTEXT_KEY])
        path = Path(value)
        if not path.is_absolute():
            path = base_path / path
            path.resolve()
        data = load_file(path)  # <-- raises raw FileNotFoundError / OSError
        return data
    raise ValidationError(f"Not supporting non file for {attr}")

When the file doesn't exist, load_file(path) raises a raw FileNotFoundError (an OS exception). Since this is not a marshmallow.ValidationError, it is not cleanly caught by UnionField._deserialize. Instead, it falls through and tries every possible schema option (ArmStr, CreateJobFileRefField, etc.), collecting all their error messages. The outer JobScheduleSchema then tries all create_job schema variants (PipelineCreateJobSchema, CommandCreateJobSchema, SparkCreateJobSchema), each of which repeats the cascade — resulting in the deeply nested, repetitive error structure.

Note: UnionField._deserialize already has a except (FileNotFoundError, TypeError) handler that catches these and adds a formatted trace message. However, the fix should be at the source — in FileRefField._deserialize — to give a clean, actionable error message immediately, rather than relying on the catch-all in UnionField.

Fix

In FileRefField._deserialize in sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py, wrap the load_file(path) call with a try/except to convert FileNotFoundError/OSError into a marshmallow.ValidationError with a clear message:

def _deserialize(self, value, attr, data, **kwargs):
    if isinstance(value, str) and not value.startswith(FILE_PREFIX):
        base_path = Path(self.context[BASE_PATH_CONTEXT_KEY])
        path = Path(value)
        if not path.is_absolute():
            path = base_path / path
            path.resolve()
        try:
            data = load_file(path)
        except (FileNotFoundError, OSError) as e:
            raise ValidationError(f"No such file or directory: {path}") from e
        return data
    raise ValidationError(f"Not supporting non file for {attr}")

This ensures marshmallow catches the error cleanly, short-circuits the union field fallback loop, and surfaces exactly one, human-readable error message to the user.

Files to Change

• sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py — Update FileRefField._deserialize to wrap load_file(path) with try/except (FileNotFoundError, OSError) and raise ValidationError.

Testing

Add or update unit tests in the schedule test file (sdk/ml/azure-ai-ml/tests/schedule/unittests/test_schedule_entity.py) to assert that when a schedule YAML references a non-existent pipeline file, the error raised is a clear ValidationError containing "No such file or directory" — and NOT the deeply nested multi-schema error blob.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: https://msdata.visualstudio.com/Vienna/_workitems/edit/3957893 can you analye the task and get the requirement here

Assistant: I'm unable to access Azure DevOps work items at https://msdata.visualstudio.com/Vienna/_workitems/edit/3957893 as it's an internal Microsoft URL that requires authentication and is not publicly accessible.

To help you analyze the task, please share the work item details directly in the chat — such as the title, description, acceptance criteria, or any relevant notes — and I'll be happy to assist from there.

User: ```
Following is the error message when you create schedule with wrong pipeline.yml path

{
"create_job": [
{
"_schema": [
"In order to specify an existing jobs, please provide either of the following prefixed with 'azureml:':\n1. The full ARM ID for the resource, e.g.azureml:/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace_name>/jobs\n2. The short-hand name of the resource registered in the workspace, eg: azureml::. For example, version 1 of the environment r...

This pull request was created from Copilot chat.