chore(copilot): test prompt blocks from a review skill - #10563
chore(copilot): test prompt blocks from a review skill#10563kirangadhave wants to merge 1 commit into
Conversation
Isolate the prompt-block templates in a review-focused skill so this experiment can determine whether skill loading produces the formatting that repository instructions alone did not. Keeping one source of truth also avoids drift between duplicated copies of the exact templates.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
All contributors have signed the CLA ✍️ ✅ |
There was a problem hiding this comment.
Pull request overview
Adds a dedicated Copilot code-review skill that instructs the reviewer to append copy‑pasteable “Prompt for AI agents” blocks to line comments and the PR overview, consolidating the experiment’s templates into a single source of truth under .github/skills/.
Changes:
- Introduces
.github/skills/code-review/SKILL.mdwith a standardized line-comment prompt block template. - Adds a standardized PR-overview prompt block template that groups violations by file and severity.
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| <file context> | ||
| <THE DIFF HUNK YOU COMMENTED ON> | ||
| </file context> |
| End every review comment you leave on a line of code with the block below. | ||
| Fill each placeholder. Reproduce the wording, the `<details>` element, and the | ||
| fence exactly as shown. Do not include the `~~~` lines. |
| Check if this issue is valid. If so, find the root cause and fix it. At <FILE_PATH>, line <LINE_NUMBER>: | ||
|
|
||
| <comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment> |
There was a problem hiding this comment.
2 issues found across 1 file
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/skills/code-review/SKILL.md">
<violation number="1" location=".github/skills/code-review/SKILL.md:11">
P3: The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the `~~~` lines," even though the only fence shown that wraps the block is the outer `~~~` fence. An executing model could interpret this as reproducing the `~~~` wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer `~~~` wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.</violation>
<violation number="2" location=".github/skills/code-review/SKILL.md:22">
P2: `<file context>` contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as `file_context` or `file-context`.</violation>
</file>
Architecture diagram
sequenceDiagram
participant Dev as Developer
participant Copilot as GitHub Copilot
participant Skill as Code Review Skill
participant Inst as Instruction Parser
participant Review as Review Generator
participant PR as Pull Request
Note over Dev,PR: Config Management Flow
Dev->>Skill: Maintain SKILL.md
Skill->>Inst: Load skill instructions
Note over Copilot,PR: Review Execution Flow
Copilot->>Skill: Request code review on PR
Skill->>Inst: Parse skill format directives
Inst-->>Skill: Return formatting rules
Skill->>Review: Generate standard line comments
Review->>Review: Format comments with prompt blocks
Review->>Review: Append details block to each comment
Review-->>PR: Post line comments with prompts
Skill->>Review: Generate overview summary
Review->>Review: Aggregate issues by file
Review->>Review: Add details block to overview
Review-->>PR: Post overview with prompt block
Note over PR,Dev: Output Verification
PR-->>Dev: Display comments and overview
alt All format blocks present
Dev->>Dev: Verify prompt blocks in comments
Dev->>Dev: Confirm overview includes blocks
else Some blocks missing
Dev->>Dev: Note unsupported instruction behavior
end
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
|
||
| <comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment> | ||
|
|
||
| <file context> |
There was a problem hiding this comment.
P2: <file context> contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as file_context or file-context.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/skills/code-review/SKILL.md, line 22:
<comment>`<file context>` contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as `file_context` or `file-context`.</comment>
<file context>
@@ -0,0 +1,56 @@
+
+<comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment>
+
+<file context>
+<THE DIFF HUNK YOU COMMENTED ON>
+</file context>
</file context>
|
|
||
| End every review comment you leave on a line of code with the block below. | ||
| Fill each placeholder. Reproduce the wording, the `<details>` element, and the | ||
| fence exactly as shown. Do not include the `~~~` lines. |
There was a problem hiding this comment.
P3: The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the ~~~ lines," even though the only fence shown that wraps the block is the outer ~~~ fence. An executing model could interpret this as reproducing the ~~~ wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer ~~~ wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/skills/code-review/SKILL.md, line 11:
<comment>The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the `~~~` lines," even though the only fence shown that wraps the block is the outer `~~~` fence. An executing model could interpret this as reproducing the `~~~` wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer `~~~` wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.</comment>
<file context>
@@ -0,0 +1,56 @@
+
+End every review comment you leave on a line of code with the block below.
+Fill each placeholder. Reproduce the wording, the `<details>` element, and the
+fence exactly as shown. Do not include the `~~~` lines.
+
+~~~
</file context>
This pull request was authored by a coding agent.
📝 Summary
This PR tests whether GitHub Copilot code review honors agent-skill instructions that append copy-pasteable prompt blocks to line comments and the pull request overview.
The previous instruction-only experiment in #10559 did not produce the blocks. The follow-up in #10562 included the same templates in both
.github/copilot-instructions.mdand a review-focused skill, and the blocks appeared. This version keeps only.github/skills/code-review/SKILL.md, making the skill the single instruction source and avoiding duplicated templates that could drift.GitHub documents both review-comment formatting changes and Pull Request Overview changes as unsupported instructions. This PR is an experiment to find out whether Copilot honors them at all; the behavior is not expected to be guaranteed.
For this PR, please include three dummy review comments so we can verify whether the prompt block appears on every line comment and in the overview.