feat: add docs-create-draft authoring skill - #113
Conversation
New skill that guides the full lifecycle of drafting a new Elastic docs page: intake questions (audience, purpose, GitHub/PR context), content type classification with free-form fallback, related-docs research via the Elastic Docs MCP, frontmatter generation (applies_to, meta_title, navigation_title), three-layer SEO title guidance, light and full review checklists, and a subagent reading test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Skill Quality Reviewdocs-create-draftOverall: A well-structured, comprehensive skill that guides an agent through the full documentation drafting workflow with clear, actionable steps. Strengths:
Issues: None found. All required frontmatter fields are present, name is kebab-case with Suggestions:
|
Skill Eval Resultsdocs-create-draft
Overall: 35/35 expectations passed (100%) The skill's instructions map cleanly to all eval expectations:
No regressions detected. The skill provides clear, deterministic instructions for all tested scenarios.
|
|
|
||
| `meta_title` is a proposed field (not yet implemented in docs-builder). Set it in the frontmatter now so it is ready when the field is available. Until then, Pattern B pages fall back to `{H1} | Elastic Docs` without the product qualifier. | ||
|
|
||
| ### Derive applies_to |
There was a problem hiding this comment.
I think we could use clearer guidance for AI regarding WHERE it should put the tags. This is a problem that I come across all the time, as AI kind of just lets them float above a section, which looks bad and makes is unclear which section of the docs the tags really apply to. Normally they either need to be inline with the paragraph's/sentence's/bullet point's beginning OR you need to create sub-sections and tag them properly.
There was a problem hiding this comment.
This should be captured in the applies_to skill (I think it already at least partly is), not diluted in various skills
There was a problem hiding this comment.
We could either make those bits reusable as fragments or just tell the skill to read the docs.
There was a problem hiding this comment.
The deployment type dimension is completely ignored here. I agree with @florent-leborgne -- this should be a separate skill (too much complexity as is).
|
|
||
| #### Style | ||
|
|
||
| **Voice and tense** — Active voice except where passive is necessary. Present tense unless future is genuinely required. Second person ("you/your"). No "I/me." Use "we" only for "we recommend." |
There was a problem hiding this comment.
"unless future is genuinely required" - this is vague for an LLM. I get what you mean here, but it would have to be more precise for AI in my opinion.
There was a problem hiding this comment.
Open to suggestions / examples!
|
|
||
| **Titles and headings** — Sentence case throughout. Describes the page distinctly from similar titles. Action-oriented where appropriate (gerund or verb for task headings). Content exists between consecutive headings (no empty headings). Consistent heading style at each level. Aim for 50–60 characters when practical. | ||
|
|
||
| **Formatting and admonitions** — Formatting is consistent. Admonitions are used sparingly (roughly three or fewer nearby) so they still stand out. No back-to-back stacked admonitions; merge into one callout or the narrative where possible. Bold for UI elements. Monospace for commands, settings, file paths, values. Oxford comma in lists of three or more. |
There was a problem hiding this comment.
I would add that stacking notes, while not forbidden, should also be used sparingly and only when absolutely necessary.
|
|
||
| **Mechanics** — Grammar, spelling, and punctuation support clarity. Simplify punctuation where it helps. | ||
|
|
||
| **Plain language and terminology** — Accessible to someone new to the topic. Jargon, acronyms, and internal terms are defined or linked on first use. Minimal wording. Informational tone. Consistent terms for ambiguous concepts. No promotional language or superlatives. |
There was a problem hiding this comment.
Maybe worth adding that terms should be defined on first use on a page, and not, for example, a section. Again, this is one of the things that are just obvious to us but not for LLMs 😄
|
|
||
| #### Technical accuracy | ||
|
|
||
| **Correctness** — SME-written or backed by an authoritative source (issue, eng review). Tested where possible. No contradictions with the docs corpus. |
There was a problem hiding this comment.
May also be backed by the PR that implemented the change I suppose (if you can find one it can be very useful)
|
|
||
| **User focus** — The change serves the user's task. Paragraphs are short. Lists, tables, and admonitions make the update scannable. Content sits in the right place on the page. | ||
|
|
||
| **Technical accuracy** — Content is technically sound and confirmed by an SME or authoritative source (issue, Slack, eng review). Does not contradict the corpus. New code samples have been tested. If the change affects version, lifecycle, or deployment scope: prerequisites remain accurate; no new permission or setup dependency is left unstated. |
There was a problem hiding this comment.
OR the corpus also needs an update - depending on the situation
|
@theletterf if you don't mind i'll bundle the skill i've been using for this same purpose over the past few months and upload it. I think this one does more or less what mine does, with the difference that I made a few iterations on mine after using it on ~100 issues - we can likely compare and take the best of both worlds or see what we can cut/adjust. |
|
@florent-leborgne Certainly more battle tested (wish I knew about it before though)! Please open a PR, I can then switch over to yours and add suggestions there — merci! |
|
We've talked about it I think :) Here it is #115 there might be a few unique things in this one or in mine worth keeping in whichever we'll ship. Mine's a bit long though I'm happy with how it works :) |
|
Hey @florent-leborgne, picking up from the Slack thread in the docs writers channel, here's a quick summary and a comparison of the two skills. Review checklist, updated H1/title guidance, and the reader subagent from this PR are the candidates to bring into #115. Entry/exit points also need reconciling. Could you annotate which parts of #113 you find worth keeping? Happy to prepare a targeted PR against #115 with those specific sections once that's clear. Differences between the two skillsWhat #115 does that #113 doesn't
What #113 has that #115 doesn't
|
|
|
||
| Write the full page content following the template from Step 2. | ||
|
|
||
| ### Title layers |
There was a problem hiding this comment.
@georgewallace Could you provide feedback on the title section? Would you rather pass it to a separate (public) skill?
|
|
||
| --- | ||
|
|
||
| ### Full review checklist |
There was a problem hiding this comment.
@yetanothertw Could you check the review step instructions?
|
|
||
| ## Step 5: Draft the page | ||
|
|
||
| Write the full page content following the template from Step 2. |
There was a problem hiding this comment.
I'd also add that the main focus is always to serve the user's task/goal (whether it's to learn about a concept or follow a step-by-step guide). It should perhaps ask for that sooner in the process, too.
The main aim is to create a logical flow to the information, follow progressive disclosure patterns, avoid duplication with existing pages, and clearly present conceptual choices.
I think you can reverse-engineer the full review checklist into guiding principles for the step when it's considering writing the content -- instead of having a dedicated review step.
|
|
||
| --- | ||
|
|
||
| ## Step 6: Self-review |
There was a problem hiding this comment.
I'm not sure the drafting process warrants a self-review section/step.
Broken down to its core, the checklist is a list of guiding principles for assessing docs quality, so I think it works better when fed as "here's how you write useful/meaningful docs" rather than "check yourself after you've already drafted stuff". Does that make sense? @georgewallace, what do you think?
| "evals": [ | ||
| { | ||
| "id": 1, | ||
| "prompt": "Draft a new page about configuring Fleet server TLS. Audience: operators with Elastic Stack experience. Purpose: walk through generating certs and applying them to Fleet server. Stack 9.0+, no serverless.", |
There was a problem hiding this comment.
This is probably a more global problem, but noting it anyway: we need to be much more specific about who our audience is. Operators, developers, etc. I get it, but what are our assumptions wrt to each of those groups? What is their technical knowledge level (are they approaching a task from a DevOps role perspective or a developer contributing to Elasticsearch)? What's their experience level with the Elastic Stack? Is the task they're trying to achieve a day 0, day 1, or later type of task?
There was a problem hiding this comment.
btw I can see we ask some of this during the intake step of the skill, I'm just advocating for having a documented strategy for alignment.
Would be great to have this documented somewhere and validated by product.
4 participants
Summary
docs-create-draft, a new authoring skill that guides the full lifecycle of drafting a new Elastic docs page from scratch.What the skill does
AskUserQuestion; fetches linked issues via the GitHub CLI; reads repotoc.ymland sibling files for conventions.search_docs,find_related_docs, andcheck_docs_coherenceto find duplication risk and crosslink candidates.applies_to,meta_title(Pattern A/B), andnavigation_titlefrom context.Agentfrom the first-time reader's perspective and reports clarity gaps back to the user.Test plan
/docs-create-draftwith a topic description and verify intake questions appearapplies_toandmeta_titleare generated correctly in frontmatter🤖 Generated with Claude Code