answer the human review round: convergence, off-ramps, per-section drafting

Signed-off-by: Nate W <natew@cncf.io>
Assisted-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: nate-double-u

specs/ai-assisted-techdocs-assessment-spec.md

@@ -4,7 +4,7 @@ title: AI-assisted TechDocs Assessment
 
 # AI-assisted TechDocs Assessment
 
-**What this is.** A working specification for an AI-assisted, human-supervised
+**What this is.** A working specification for an AI-assisted, expert-supervised
 CNCF TechDocs assessment workflow, in two parts. Part I is requirements: the
 outcomes the system must produce and the boundaries it must never cross. Part II
 is the operational specification: how those requirements bind to concrete
@@ -121,7 +121,9 @@ the labor. AI drafts, humans decide, all in the open.
   boundary), not a named product.
 - P-4: Reproducible and auditable. Every claim in a deliverable traces to a
   cited, committed source. Prompts, configuration, and any data-generating steps
-  are versioned and re-runnable.
+  are versioned and re-runnable. Reproducible means convergent, not identical:
+  two honest runs over the same material surface the same major findings, though
+  details and wording differ (section 15).
 
 ### Hard constraints (the "never" list)
 
@@ -151,7 +153,9 @@ the labor. AI drafts, humans decide, all in the open.
   already given.
 - HC-5: Provenance required. Quantitative claims (counts, link status,
   inventories) must be generated by a committed, re-runnable step and cite their
-  source; no unverifiable figures in a deliverable.
+  source; no unverifiable figures in a deliverable. Citation is the working
+  anti-hallucination control: a claim that names its source can be checked, and
+  the verifier (section 4) depends on those handholds.
 - HC-6: Disclose AI involvement. The assessed project is told, up front in the
   intake and in a header on every deliverable, that the analysis was AI-drafted
   and human-reviewed. No project should discover this after the fact.
@@ -220,11 +224,17 @@ Each phase runs the same six steps:
    becomes eligible once the previous one is merged (phase C directly after A
    when phase B is skipped; section 6).
 2. Accept. A technical writer triages and explicitly accepts the request, for
-   every phase, not just A; eligibility alone does not start work (P-1).
+   every phase, not just A; eligibility alone does not start work (P-1). Triage
+   can equally decline (section 12); acceptance is a choice, not a formality.
 3. Draft. The drafter produces the deliverable as a draft PR. When the phase's
    drafter takes data-collection outputs as input (section 14), delegation
    produces them first: they are generated deterministically at session start,
    before any model runs, and committed with the draft as its evidence (HC-5).
+   Drafting may be decomposed by criteria section: successive delegations, each
+   free to use a different model or effort (section 14), accumulate into the
+   same draft PR. Decomposition changes how the draft is produced, not what is
+   reviewed: the deliverable stays whole, and review (step 4) reads it as one
+   document.
 4. Review. The verifier's fact-check pass runs first; the reviewer may skip it
    deliberately, recording why (section 12). The reviewer then refines the draft
    in conversation with the drafter, verifies findings against source (HC-7),
@@ -237,9 +247,10 @@ Each phase runs the same six steps:
 
 Failure path: if a draft is bad enough that fixing it would cost more than
 starting over, the reviewer may discard it and restart or write by hand rather
-than sink time into editing. Repeated failure is escalated to the
-administrator/platform owner, who may abort the run. An assessment must never
-end up slower than doing it by hand.
+than sink time into editing. Each discarded draft is a decision point: restart,
+hand-write, or stop; retries are chosen, never automatic. Repeated failure is
+escalated to the administrator/platform owner, who may abort the run. An
+assessment must never end up slower than doing it by hand.
 
 ## 6. Phases and deliverables
 
@@ -253,7 +264,8 @@ end up slower than doing it by hand.
   methodology's time bound (a few hours to a couple of days at most for someone
   experienced with the project and comfortable writing), plus an umbrella/index
   file that lists them. Effort estimates are the agent's first pass and are
-  coherence-checked by the reviewer; agent estimates are not reliable on their own.
+  coherence-checked by the reviewer; agent estimates are not reliable on their
+  own.
 
   Proposed methodology change. The current method delivers the backlog as one
   `_PROJECT_-issues.md` file (`howto.md`, `templates/issues-list.md`,
@@ -276,8 +288,10 @@ Scope note: phase one produces plans (assessment, implementation, backlog), not
 documentation changes, and the backlog lands as files in cncf/techdocs rather
 than as issues where writers work (NG-2, HC-3). Until the optional filing tool
 exists (section 17), the reviewer and stakeholders route the backlog to writers
-by hand. The pilot therefore measures output, not documentation outcomes, a
-distinction we state so we don't mistake one for the other.
+by hand. The pilot, and any later evaluation of this process, measures output
+(quality, time, and cost to produce), not documentation outcomes, which turn on
+contributor effort beyond this process's control; we state the distinction so we
+don't mistake one for the other.
 
 ## 7. Inputs: the assessment request
 
@@ -289,10 +303,11 @@ sections, plus maturity and contacts).
 
 Filing a request does not start an assessment: it gathers context, and a
 technical writer triages and explicitly accepts it to begin Phase A (NG-5, P-1).
-A request can also be filed by the assessing team rather than the project, and
-HC-6's up-front promise must not depend on who filed: phase A is not delegated
-until the project contacts named in the request have been notified on the intake
-issue.
+Filing also obligates no one: the program may decline the request, and the
+project may withdraw at any point in the lifecycle (section 12). A request can
+also be filed by the assessing team rather than the project, and HC-6's up-front
+promise must not depend on who filed: phase A is not delegated until the project
+contacts named in the request have been notified on the intake issue.
 
 Open question: how this relates to the existing CNCF service desk and TechDocs
 assistance-program intake, so we don't create a competing front door.
@@ -522,13 +537,16 @@ build.
   the intake issue: an unaccepted eligible phase is the stall mode, so the
   umbrella shows when work actually started and by whom. The triage verdict has
   a counterpart: a writer declines an unaccepted request with
-  `/decline <reason>`, under the same authorization as `/accept`; the workflow
-  labels it `triage/declined`, posts the reason, and closes the issue as not
-  planned. Declining the intake means the assessment never starts. Declining a
-  phase B or C tracking issue ends the assessment there, noted on the intake
-  issue, which the workflow closes as not planned, so an open intake still means
-  work in flight. A project's request to stop is honored the same way: the
-  writer records a decline rather than leaving the request to sit unanswered.
+  `/decline [reason]`, under the same authorization as `/accept`; the workflow
+  labels it `triage/declined`, posts the reason when one is given, and closes
+  the issue as not planned. The reason is optional: most declines should carry
+  one, but a sensitive reason can go to the project privately instead of into
+  the public record. Declining the intake means the assessment never starts.
+  Declining a phase B or C tracking issue ends the assessment there, noted on
+  the intake issue, which the workflow closes as not planned, so an open intake
+  still means work in flight. A project's request to stop is honored the same
+  way: the writer records a decline rather than leaving the request to sit
+  unanswered.
 - Draft (step 3): a draft pull request. The agent works on its own branch in
   cncf/techdocs and opens a draft PR linked to the tracking issue, carrying the
   provenance block (section 15).
@@ -991,13 +1009,13 @@ second assessment overlaps it (section 12).
   or selection needs the Agents panel (section 12); how the assessment's target
   project reaches the setup steps that run collection, or whether collection
   needs a different trigger (section 13); whether a full phase A draft, with its
-  setup-step collection sharing the envelope, fits the session cap or the
-  drafting needs decomposing (section 11); whether effort level can be pinned in
-  a profile or anywhere else, the one model-choice question the documentation
-  leaves open (section 14); and whether the cloud-agent configuration read
-  endpoint, in public preview as of this writing, returns the fields the
-  snapshot needs and what credential the snapshot script must hold to call it
-  (section 13).
+  setup-step collection sharing the envelope, fits the session cap in a single
+  delegation or leans on the per-section decomposition section 5 allows (section
+  11); whether effort level can be pinned in a profile or anywhere else, the one
+  model-choice question the documentation leaves open (section 14); and whether
+  the cloud-agent configuration read endpoint, in public preview as of this
+  writing, returns the fields the snapshot needs and what credential the
+  snapshot script must hold to call it (section 13).
 - Filing issues into project repos. A separate, opt-in tool to create the
   backlog issues in a project's own repository (NG-2). Out of scope for phase
   one. It would be human-run with its own credential and the project's explicit
@@ -1008,6 +1026,14 @@ second assessment overlaps it (section 12).
   front door. Related: the contribute.cncf.io site needs a page for projects on
   requesting an assessment and what to expect, including the AI disclosure,
   updated as this process changes.
+- Cost and usage reporting. Recording per-assessment expert hours and AI usage
+  (credit or token draw) so G-1's compression claim can be judged against real
+  numbers rather than impressions. Out of scope for the MVP.
+- Template provenance. Propose adding the provenance block skeleton (section 15)
+  to the methodology's templates as body content, under the same P-2 ordering as
+  the section 6 issues-split proposal, and define what the template front
+  matter's `author:` field carries on an AI-drafted deliverable. Until the
+  methodology accepts it, the drafter's instructions carry the block format.
 - AI-readiness criteria. Whether and how the criteria evolve for AI consumption
   (for example llms.txt), tracked in PR #357.
 - Assessment-quality rubric. Define the meta-rubric that scores an assessment's