Skip to content

chore(copilot): test prompt blocks from a review skill - #10563

Closed
kirangadhave wants to merge 1 commit into
mainfrom
kg/better-copilot-comments
Closed

chore(copilot): test prompt blocks from a review skill#10563
kirangadhave wants to merge 1 commit into
mainfrom
kg/better-copilot-comments

Conversation

@kirangadhave

@kirangadhave kirangadhave commented Aug 15, 2026

Copy link
Copy Markdown
Member

This pull request was authored by a coding agent.

📝 Summary

This PR tests whether GitHub Copilot code review honors agent-skill instructions that append copy-pasteable prompt blocks to line comments and the pull request overview.

The previous instruction-only experiment in #10559 did not produce the blocks. The follow-up in #10562 included the same templates in both .github/copilot-instructions.md and a review-focused skill, and the blocks appeared. This version keeps only .github/skills/code-review/SKILL.md, making the skill the single instruction source and avoiding duplicated templates that could drift.

GitHub documents both review-comment formatting changes and Pull Request Overview changes as unsupported instructions. This PR is an experiment to find out whether Copilot honors them at all; the behavior is not expected to be guaranteed.

For this PR, please include three dummy review comments so we can verify whether the prompt block appears on every line comment and in the overview.

Isolate the prompt-block templates in a review-focused skill so this
experiment can determine whether skill loading produces the formatting that
repository instructions alone did not. Keeping one source of truth also
avoids drift between duplicated copies of the exact templates.
@vercel

vercel Bot commented Aug 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview Aug 15, 2026 12:05am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kirangadhave
kirangadhave marked this pull request as ready for review August 15, 2026 00:06
Copilot AI lite review requested due to automatic review settings August 15, 2026 00:06
@kirangadhave kirangadhave added the internal A refactor or improvement that is not user facing label Aug 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a dedicated Copilot code-review skill that instructs the reviewer to append copy‑pasteable “Prompt for AI agents” blocks to line comments and the PR overview, consolidating the experiment’s templates into a single source of truth under .github/skills/.

Changes:

  • Introduces .github/skills/code-review/SKILL.md with a standardized line-comment prompt block template.
  • Adds a standardized PR-overview prompt block template that groups violations by file and severity.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +22 to +24
<file context>
<THE DIFF HUNK YOU COMMENTED ON>
</file context>
Comment on lines +9 to +11
End every review comment you leave on a line of code with the block below.
Fill each placeholder. Reproduce the wording, the `<details>` element, and the
fence exactly as shown. Do not include the `~~~` lines.
Comment on lines +18 to +20
Check if this issue is valid. If so, find the root cause and fix it. At <FILE_PATH>, line <LINE_NUMBER>:

<comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/skills/code-review/SKILL.md">

<violation number="1" location=".github/skills/code-review/SKILL.md:11">
P3: The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the `~~~` lines," even though the only fence shown that wraps the block is the outer `~~~` fence. An executing model could interpret this as reproducing the `~~~` wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer `~~~` wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.</violation>

<violation number="2" location=".github/skills/code-review/SKILL.md:22">
P2: `<file context>` contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as `file_context` or `file-context`.</violation>
</file>
Architecture diagram
sequenceDiagram
    participant Dev as Developer
    participant Copilot as GitHub Copilot
    participant Skill as Code Review Skill
    participant Inst as Instruction Parser
    participant Review as Review Generator
    participant PR as Pull Request

    Note over Dev,PR: Config Management Flow
    
    Dev->>Skill: Maintain SKILL.md
    Skill->>Inst: Load skill instructions

    Note over Copilot,PR: Review Execution Flow
    
    Copilot->>Skill: Request code review on PR
    Skill->>Inst: Parse skill format directives
    Inst-->>Skill: Return formatting rules
    Skill->>Review: Generate standard line comments
    Review->>Review: Format comments with prompt blocks
    Review->>Review: Append details block to each comment
    Review-->>PR: Post line comments with prompts
    
    Skill->>Review: Generate overview summary
    Review->>Review: Aggregate issues by file
    Review->>Review: Add details block to overview
    Review-->>PR: Post overview with prompt block
    
    Note over PR,Dev: Output Verification
    
    PR-->>Dev: Display comments and overview
    
    alt All format blocks present
        Dev->>Dev: Verify prompt blocks in comments
        Dev->>Dev: Confirm overview includes blocks
    else Some blocks missing
        Dev->>Dev: Note unsupported instruction behavior
    end
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic


<comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment>

<file context>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: <file context> contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as file_context or file-context.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/skills/code-review/SKILL.md, line 22:

<comment>`<file context>` contains a space, which is not a valid XML/HTML tag name. Since this block is meant to be machine-consumable, use a valid tag name such as `file_context` or `file-context`.</comment>

<file context>
@@ -0,0 +1,56 @@
+
+<comment severity="<low|medium|high>"><THE TEXT OF YOUR REVIEW COMMENT></comment>
+
+<file context>
+<THE DIFF HUNK YOU COMMENTED ON>
+</file context>
</file context>


End every review comment you leave on a line of code with the block below.
Fill each placeholder. Reproduce the wording, the `<details>` element, and the
fence exactly as shown. Do not include the `~~~` lines.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the ~~~ lines," even though the only fence shown that wraps the block is the outer ~~~ fence. An executing model could interpret this as reproducing the ~~~ wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer ~~~ wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/skills/code-review/SKILL.md, line 11:

<comment>The guidance is self-contradictory: it says to reproduce "the fence exactly as shown" but then says "Do not include the `~~~` lines," even though the only fence shown that wraps the block is the outer `~~~` fence. An executing model could interpret this as reproducing the `~~~` wrapper. Clarify that the copy-pasteable block is the inner `````text fence and that the outer `~~~` wrapper is only scaffolding. Apply the same fix in the "Overview comment format" section.</comment>

<file context>
@@ -0,0 +1,56 @@
+
+End every review comment you leave on a line of code with the block below.
+Fill each placeholder. Reproduce the wording, the `<details>` element, and the
+fence exactly as shown. Do not include the `~~~` lines.
+
+~~~
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal A refactor or improvement that is not user facing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants