Sync .github directory with azure-sdk-tools for PR 15062 by azure-sdk · Pull Request #48773 · Azure/azure-sdk-for-java

azure-sdk · 2026-04-10T18:51:40Z

Sync .github directory with azure-sdk-tools for PR Azure/azure-sdk-tools#15062 See eng/common workflow

Copilot

Pull request overview

Syncs the .github/skills directory with the upstream azure-sdk-tools changes (PR 15062) by adding multiple “utility skill” definitions plus their task/eval suites, enabling standardized Copilot/MCP-assisted workflows (release, release planning, pipeline troubleshooting, local generation, APIView feedback).

Changes:

Add 5 common workflow skills (sdk-release, prepare-release-plan, pipeline-troubleshooting, generate-sdk-locally, apiview-feedback-resolution) with SKILL.md guidance and references/fixtures.
Add task YAMLs for each skill (happy-path, edge cases, negative/anti-trigger).
Add evaluation configs (eval.yaml, evals/*) to validate trigger behavior and basic response constraints.

Reviewed changes

Copilot reviewed 60 out of 60 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
.github/skills/azsdk-common-sdk-release/tasks/release-trigger-001.yaml	Task case for triggering release pipeline
.github/skills/azsdk-common-sdk-release/tasks/release-readiness-001.yaml	Task case for readiness-only flow
.github/skills/azsdk-common-sdk-release/tasks/release-negative-001.yaml	Negative task case (should not trigger)
.github/skills/azsdk-common-sdk-release/tasks/release-basic-001.yaml	Happy-path full release flow task
.github/skills/azsdk-common-sdk-release/SKILL.md	Skill definition and usage guidance
.github/skills/azsdk-common-sdk-release/evals/trigger_tests.yaml	Trigger/anti-trigger prompt set
.github/skills/azsdk-common-sdk-release/evals/tasks/basic-trigger.yaml	Basic trigger eval task
.github/skills/azsdk-common-sdk-release/evals/tasks/anti-trigger.yaml	Anti-trigger eval task
.github/skills/azsdk-common-sdk-release/evals/eval.yaml	Evals suite config
.github/skills/azsdk-common-sdk-release/eval.yaml	Evaluation suite config (non-evals)
.github/skills/azsdk-common-prepare-release-plan/tasks/should-not-trigger.yaml	Negative task case to avoid activating on “release package” requests
.github/skills/azsdk-common-prepare-release-plan/tasks/link-sdk-prs.yaml	Task case for linking SDK PRs to a release plan
.github/skills/azsdk-common-prepare-release-plan/tasks/edge-case.yaml	Edge-case validation task for package naming formats
.github/skills/azsdk-common-prepare-release-plan/tasks/basic-usage.yaml	Basic usage task for creating a release plan
.github/skills/azsdk-common-prepare-release-plan/SKILL.md	Skill definition and usage guidance
.github/skills/azsdk-common-prepare-release-plan/references/release-plan-details.md	Detailed reference steps and constraints
.github/skills/azsdk-common-prepare-release-plan/fixtures/tspconfig.yaml	Fixture TypeSpec config used for examples/tests
.github/skills/azsdk-common-prepare-release-plan/evals/trigger_tests.yaml	Trigger/anti-trigger prompt set
.github/skills/azsdk-common-prepare-release-plan/evals/tasks/basic-trigger.yaml	Basic trigger eval task
.github/skills/azsdk-common-prepare-release-plan/evals/tasks/anti-trigger.yaml	Anti-trigger eval task
.github/skills/azsdk-common-prepare-release-plan/evals/eval.yaml	Evals suite config
.github/skills/azsdk-common-prepare-release-plan/eval.yaml	Evaluation suite config (non-evals)
.github/skills/azsdk-common-pipeline-troubleshooting/tasks/should-not-trigger.yaml	Negative task case (TypeSpec authoring shouldn’t trigger)
.github/skills/azsdk-common-pipeline-troubleshooting/tasks/local-reproduction.yaml	Task case for local reproduction guidance
.github/skills/azsdk-common-pipeline-troubleshooting/tasks/edge-case.yaml	Edge-case task for multi-language failures
.github/skills/azsdk-common-pipeline-troubleshooting/tasks/basic-usage.yaml	Basic usage task for pipeline failure analysis
.github/skills/azsdk-common-pipeline-troubleshooting/SKILL.md	Skill definition and usage guidance
.github/skills/azsdk-common-pipeline-troubleshooting/references/failure-patterns.md	Reference guide for common pipeline failure patterns
.github/skills/azsdk-common-pipeline-troubleshooting/evals/trigger_tests.yaml	Trigger/anti-trigger prompt set
.github/skills/azsdk-common-pipeline-troubleshooting/evals/tasks/basic-trigger.yaml	Basic trigger eval task
.github/skills/azsdk-common-pipeline-troubleshooting/evals/tasks/anti-trigger.yaml	Anti-trigger eval task
.github/skills/azsdk-common-pipeline-troubleshooting/evals/eval.yaml	Evals suite config
.github/skills/azsdk-common-pipeline-troubleshooting/eval.yaml	Evaluation suite config (non-evals)
.github/skills/azsdk-common-generate-sdk-locally/tasks/rename-client.yaml	Task case for TypeSpec customization to rename client/model
.github/skills/azsdk-common-generate-sdk-locally/tasks/hide-operation.yaml	Task case for hiding internal operations from SDK surface
.github/skills/azsdk-common-generate-sdk-locally/tasks/full-workflow.yaml	Task case for full local generation/build/test workflow
.github/skills/azsdk-common-generate-sdk-locally/tasks/edge-case.yaml	Edge-case task for build failure recovery
.github/skills/azsdk-common-generate-sdk-locally/tasks/customization-workflow.yaml	Task case for customization drift/duplicate field conflicts
.github/skills/azsdk-common-generate-sdk-locally/tasks/breaking-changes.yaml	Task case for resolving breaking changes after spec updates
.github/skills/azsdk-common-generate-sdk-locally/tasks/basic-usage.yaml	Basic usage task for local generation (Python example)
.github/skills/azsdk-common-generate-sdk-locally/tasks/anti-trigger.yaml	Anti-trigger task (pipeline-based generation)
.github/skills/azsdk-common-generate-sdk-locally/tasks/analyzer-errors.yaml	Task case for .NET analyzer-driven renames/customizations
.github/skills/azsdk-common-generate-sdk-locally/SKILL.md	Skill definition and usage guidance
.github/skills/azsdk-common-generate-sdk-locally/references/sdk-repos.md	Reference mapping of languages to SDK repos
.github/skills/azsdk-common-generate-sdk-locally/references/customization-workflow.md	Reference describing 2-phase customization workflow
.github/skills/azsdk-common-generate-sdk-locally/fixtures/tspconfig.yaml	Fixture TypeSpec config used for examples/tests
.github/skills/azsdk-common-generate-sdk-locally/eval.yaml	Evaluation suite config
.github/skills/azsdk-common-apiview-feedback-resolution/tasks/should-not-trigger.yaml	Negative task case (TypeSpec project creation)
.github/skills/azsdk-common-apiview-feedback-resolution/tasks/no-feedback.yaml	Task case for “no APIView comments found”
.github/skills/azsdk-common-apiview-feedback-resolution/tasks/edge-case.yaml	Edge-case task for feedback requiring TypeSpec changes
.github/skills/azsdk-common-apiview-feedback-resolution/tasks/basic-usage.yaml	Basic usage task for retrieving APIView comments
.github/skills/azsdk-common-apiview-feedback-resolution/SKILL.md	Skill definition and usage guidance
.github/skills/azsdk-common-apiview-feedback-resolution/references/feedback-resolution-steps.md	Reference steps for categorizing/resolving feedback
.github/skills/azsdk-common-apiview-feedback-resolution/fixtures/apiview-comment.json	Fixture APIView comment payload
.github/skills/azsdk-common-apiview-feedback-resolution/evals/trigger_tests.yaml	Trigger/anti-trigger prompt set (incl. URL-only prompts)
.github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/url-trigger.yaml	Eval task for APIView URL-only trigger
.github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/basic-trigger.yaml	Basic trigger eval task
.github/skills/azsdk-common-apiview-feedback-resolution/evals/tasks/anti-trigger.yaml	Anti-trigger eval task
.github/skills/azsdk-common-apiview-feedback-resolution/evals/eval.yaml	Evals suite config
.github/skills/azsdk-common-apiview-feedback-resolution/eval.yaml	Evaluation suite config (non-evals)

Copilot · 2026-04-10T19:00:22Z

+graders:
+- type: text
+  name: no_fatal_errors
+  config:
+    regex_not_match:


This eval config isn’t valid YAML: the list item under graders: isn’t indented, and the regex_not_match patterns are also aligned as if they were a sibling key. YAML parsers will treat this as a top-level sequence / malformed mapping. Indent the grader item(s) under graders: and indent the regex_not_match list under that key.

Copilot · 2026-04-10T19:00:23Z

+graders:
+- type: text
+  name: no_fatal_errors
+  config:
+    regex_not_match:


This eval config isn’t valid YAML: the list item under graders: isn’t indented, and the regex_not_match patterns are also aligned as if they were a sibling key. Indent the grader item(s) under graders: and indent the regex_not_match list under that key so the file can be parsed.

Copilot · 2026-04-10T19:00:23Z

+- type: text
+  name: no_fatal_errors
+  config:
+    regex_not_match:
+    - (?i)fatal error
+    - (?i)unhandled exception
+    - '(?i)panic:'


This eval config isn’t valid YAML: the grader list item is not indented under graders:, and the regex_not_match entries are not nested as a YAML list under that key. This will fail parsing in most YAML loaders. Please fix the indentation for the graders section.

Suggested change

- type: text

name: no_fatal_errors

config:

regex_not_match:

- (?i)fatal error

- (?i)unhandled exception

- '(?i)panic:'

- type: text

name: no_fatal_errors

config:

regex_not_match:

- (?i)fatal error

- (?i)unhandled exception

- '(?i)panic:'

Copilot · 2026-04-10T19:00:23Z

+- type: text
+  name: no_fatal_errors
+  config:
+    regex_not_match:
+    - (?i)fatal error
+    - (?i)unhandled exception
+    - '(?i)panic:'


This eval config isn’t valid YAML: the grader list item starts at column 0 instead of being indented under graders:, and the regex_not_match values are not nested as a list under that key. Please indent the graders section so the file parses correctly.

Suggested change

- type: text

name: no_fatal_errors

config:

regex_not_match:

- (?i)fatal error

- (?i)unhandled exception

- '(?i)panic:'

- type: text

name: no_fatal_errors

config:

regex_not_match:

- (?i)fatal error

- (?i)unhandled exception

- '(?i)panic:'

Copilot · 2026-04-10T19:00:24Z

+    config:
+      regex_not_match:
+        - (?i)error occurred
+        - (?i)failed


The no_fatal_errors grader forbids the word failed (case-insensitive). That’s likely to cause false failures for this skill, since normal troubleshooting guidance often includes phrases like “build failed” or “tests failed”. Consider aligning this grader with the others (e.g., only block truly fatal/unhandled error patterns) or narrowing the regexes to avoid rejecting legitimate responses.

Suggested change

- (?i)failed

- (?i)fatal error

- (?i)unhandled exception

Copilot · 2026-04-10T19:00:24Z

+- Check if tests need playback recording updates
+- Verify test fixtures are present and valid


This guidance suggests updating/re-recording test playbacks as a generic remediation. For Azure SDK repos this is usually not an acceptable first-line fix (it can mask real regressions and introduces churn). Consider replacing this with diagnostics-first guidance (identify why playback diverged, validate sanitizers/matchers, update test logic) and only mention recording refresh as a last resort when service behavior legitimately changed.

Copilot · 2026-04-10T19:00:24Z

+| ----------------------------- | ------------------------------ | ---------------------------------- |
+| Missing types/models          | TypeSpec compilation issue     | Fix TypeSpec, regenerate           |
+| Breaking change detected      | API surface changed            | Add `@clientName` or revert change |
+| Test playback failure         | Recorded responses outdated    | Re-record test sessions            |


The failure-pattern table recommends “Re-record test sessions” as the fix for playback failures. This is generally not a good default recommendation (it can hide regressions and creates unnecessary churn). Prefer guidance to diagnose the mismatch first and reserve recording refresh for cases where the live service behavior legitimately changed.

The generate-sdk-locally shared skill (synced via PR Azure#48773) and Azure SDK MCP tools already handle generation, build, test, changelog, and versioning. The search package skill was re-documenting these steps, creating overlap. Refactored to: - Defer generation/build/test to generate-sdk-locally skill and MCP tools - Keep ONLY search-specific content: error categorization table, customization patterns, ServiceVersion management, breaking change detection, import cleanup - Add explicit 'use generate-sdk-locally for generation/build/test' directive Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…s have been tried Avoids duplication with generate-sdk-locally (PR Azure#48773) which already runs azsdk_customized_code_update on build failures. This skill covers what those tools can't solve — search-specific patterns. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename 5 shared skills to use the azsdk-common- prefix so the sync-.github.yml pipeline can pattern-match and distribute them to all subscribed Azure SDK language repos. Renames: generate-sdk-locally → azsdk-common-generate-sdk-locally apiview-feedback-resolution → azsdk-common-apiview-feedback-resolution pipeline-troubleshooting → azsdk-common-pipeline-troubleshooting prepare-release-plan → azsdk-common-prepare-release-plan sdk-release → azsdk-common-sdk-release Updated: - SKILL.md name: fields in all 5 skills - All eval.yaml and trigger_tests.yaml skill/name references - .github/skills/README.md links and naming convention docs - sensei SKILL.md example reference - 8 instruction files in eng/common/instructions/azsdk-tools/ with Related Skill cross-references pointing to new skill paths - copilot/sdk-release.instructions.md with skill cross-reference Relates to #15049 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Delete 7 instruction files whose content is now fully covered by skills: - local-sdk-workflow.instructions.md → azsdk-common-generate-sdk-locally - create-release-plan.instructions.md → azsdk-common-prepare-release-plan - sdk-details-in-release-plan.instructions.md → azsdk-common-prepare-release-plan - verify-namespace-approval.instructions.md → azsdk-common-prepare-release-plan - check-package-readiness.instructions.md → azsdk-common-sdk-release - check-package-validation.instructions.md → azsdk-common-generate-sdk-locally - copilot/sdk-release.instructions.md → azsdk-common-sdk-release Skills updated with all unique details from instruction files: - generate-sdk-locally: added commit checkpoints, language selection, config file identification, SDK project path, metadata update tools - prepare-release-plan: added critical LLM instructions (NextSteps field), Release Planner Tool link, package name extraction rules per language, valid/invalid examples table, JSON structure, namespace approval flow - sdk-release: added readiness check details, interaction flow pattern, do-not-ask-for-PR/plan guard, case-sensitive language note typespec-to-sdk orchestrator updated to reference skills directly. Removed all related_skill frontmatter and blockquotes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The previous commit incorrectly rewrote 3 SKILL.md files from scratch. This restores the original content authored by the tools team, with only the name: field updated to use the azsdk-common- prefix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Agent-Logs-Url: https://github.com/Azure/azure-sdk-tools/sessions/b9198c9e-83ff-4bd0-8ae1-467763121903 Co-authored-by: l0lawrence <100643745+l0lawrence@users.noreply.github.com>

ronniegeraghty · 2026-04-20T17:23:08Z

Closing — source PR Azure/azure-sdk-tools#15062 was closed and superseded by Azure/azure-sdk-tools#15210. New sync PRs will be generated from #15210 when it lands.

azure-sdk requested a review from a team as a code owner April 10, 2026 18:51

azure-sdk requested review from Copilot and ronniegeraghty April 10, 2026 18:51

azure-sdk added EngSys This issue is impacting the engineering system. Central-EngSys This issue is owned by the Engineering System team. labels Apr 10, 2026

azure-sdk assigned ronniegeraghty Apr 10, 2026

Copilot AI reviewed Apr 10, 2026

View reviewed changes

ronniegeraghty mentioned this pull request Apr 10, 2026

[AzSDK Agent] E2E Validation: Java — Full workflow with new shared skills Azure/azure-sdk-tools#15054

Closed

5 tasks

samvaity approved these changes Apr 14, 2026

View reviewed changes

azure-sdk force-pushed the sync-.github-squad/15049-sync-shared-skills-15062 branch from c7ef2ca to 7187736 Compare April 14, 2026 23:42

samvaity mentioned this pull request Apr 15, 2026

Add YAML lint to skill CI and fix eval.yaml indentation Azure/azure-sdk-tools#15148

Open

ronniegeraghty and others added 4 commits April 17, 2026 20:47

Fix cspell unknown word 'customiz' in customization-workflow.yaml

892c13d

Agent-Logs-Url: https://github.com/Azure/azure-sdk-tools/sessions/b9198c9e-83ff-4bd0-8ae1-467763121903 Co-authored-by: l0lawrence <100643745+l0lawrence@users.noreply.github.com>

azure-sdk force-pushed the sync-.github-squad/15049-sync-shared-skills-15062 branch from 7187736 to 892c13d Compare April 17, 2026 20:47

ronniegeraghty closed this Apr 20, 2026

azure-sdk deleted the sync-.github-squad/15049-sync-shared-skills-15062 branch April 26, 2026 09:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sync .github directory with azure-sdk-tools for PR 15062#48773

Sync .github directory with azure-sdk-tools for PR 15062#48773
azure-sdk wants to merge 4 commits into
mainfrom
sync-.github-squad/15049-sync-shared-skills-15062

azure-sdk commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

ronniegeraghty commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		- Check if tests need playback recording updates
		- Verify test fixtures are present and valid

Conversation

azure-sdk commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

ronniegeraghty commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants