Task: Track documentation contributions and derive the Documenter achievement from them

[herzog0]

Context

The badges catalogue - the set of achievement and badge definitions seeded into the database - already
contains a Documentation achievement and a Documenter badge, with thresholds of 1 / 3 / 7 / 19 / 50.
It has no automatic source, so today it can only be granted by hand in the admin.

The reason it has none is that nothing in the site records who wrote documentation. Commits are
imported per library and stored on the Commit model, but only the author, message, date, sha and
whether it was a merge are kept. No file paths are stored, so a commit that rewrote a library's
Antora docs is indistinguishable from one that changed a header.

The spike concluded that the cheapest fix is at the point of ingestion: the commit importer already
clones each library and shells out to git log, so asking git for per-file statistics in the same
call costs one flag and no additional GitHub API traffic. Each commit can then be classified as
touching documentation or not, and the result stored alongside the commit. That turns Documenter into
an ordinary automatic source, counting doc-touching commits the same way the Code Commits achievement
counts all commits.

Two terms from the engine ticket are used below, because the tests turn on them. A backfill walks
a source and grants whatever is missing, never removing anything, which is what makes it safe to run
unattended. A reconcile walks the same source in both directions and also removes grants the
source no longer supports.

Scope

The commit import

1. Ask git for per-file statistics in the existing git log call in
   get_commit_data_for_repo_versions(). Merge commits produce no per-file output and so exclude
   themselves, which matches the intent: a merge did not write the docs.
2. Keep the statistics out of the commit message. The parser currently captures everything
   between one commit header and the next as the message, so the new lines would be appended to
   every Commit.message in the database. The parse has to separate the two, and a test should pin
   that a message round-trips unchanged with the flag on.
3. Count the files that qualify as documentation, per the rules below, and carry the count on
   ParsedCommit.

Persistence

4. Store the count on Commit and thread it through handle_commit() into the bulk write in
   update_commits(). Migration required.

5. Add it to the write's update_fields, or existing rows never gain the value. In its
   non-destructive mode the importer updates matched rows in place and keeps their ids, so that path
   heals history without disturbing any grant already pointing at a commit. That is the mode the
   weekly release pipeline uses.

6. Discard the grants sourced from a commit before the destructive mode deletes it.
   LibraryUpdater.update_commits() takes a clean argument that deletes every Commit row for the
   library before re-importing, so the rows come back with new ids. A grant points at its source row
   by table and id with no database-level link back, so those grants are left pointing at ids that no
   longer exist. That is worse than a wrong count: a reconcile can only remove a grant it can match
   against what an iterator yields today, and a grant whose source is gone matches nothing, so it
   counts toward a threshold permanently and no sweep can ever clean it up.

   This is not an expert opt-in. import_commits --clean is the opt-in path, but
   libraries/admin.py hardcodes clean=True for the "Update Commits" button on the commit
   changelist, so every press wipes and reinserts the whole table. It is also one of the
   unconverted legacy GET-triggered admin views, so it fires from a bare link with no confirmation
   step.

   discard_source_achievements already exists for exactly this and the review importer uses it.
   Wiring it into the delete also closes the same hole for Code Commits, which is sourced from the
   same table and has it today.

The achievement

7. A source iterator - the function the ingestion engine walks for one achievement, yielding one
   (member, thing) pair for every grant that should exist - yielding (member, commit) for every
   commit that touched documentation and whose author is linked to a member. Commit authors are
   CommitAuthor rows and only some are claimed; an unlinked one is skipped rather than guessed at,
   which is what the Code Commits iterator already does.
8. Register it against its achievement, which is also what makes it selectable wherever a single
   source can be named, in both the commands and the admin buttons. That list must stay derived from
   what is actually registered rather than restated somewhere it can drift.
9. Stop describing Documentation as having no automatic source wherever the code says so today.
10. Tests: the classifier against every line of the include and ignore lists below, the iterator
    skipping unlinked authors, and one end-to-end case where a doc commit produces a grant and the
    Documenter badge while a code-only commit produces neither.

Doc-path classification rules

The rules are the spike's, restated here so this ticket stands alone.

Counts as documentation:

• any path under doc/ or docs/ at any depth, including images and other media
• *.adoc, *.qbk, *.rst and *.md anywhere, except a README.md at the repository root

Ignored even inside a doc tree:

• any *.json, meta/ included
• CI and build config: anything under .github/, plus *.yml, *.yaml, *.cmake, Jamfile*
  and *.jam
• generated output directories such as doc/html/, which are build artifacts and would inflate
  counts by orders of magnitude

Two shapes of git's per-file output need handling and are worth a test each: a rename is reported as
a single path containing a brace expansion rather than two paths, and a binary file reports no line
counts at all. Neither should be miscounted or crash the parse.

Out of scope

• Backfilling doc data for repositories that are not re-imported. Rows heal on the next import of
  that library; when to run one across every library is an operational decision and its cost is
  discussed below.
• Changing what the "Update Commits" button does. Whether it should stop deleting rows is a question
  about existing commit-import behaviour, raised below but not settled here.
• Documentation that does not arrive as a commit to a library repository: the website-v2 docs, the
  website-v2-docs repository, and anything hosted outside GitHub.
• Any UI. The badge already renders wherever badges render.
• Weighting a commit by how much documentation it changed. The count is stored so that this stays
  possible later, not so it happens now.
• Any change to the achievement's thresholds. They are already seeded, and retuning one is a data
  migration rather than an edit here.

Acceptance criteria

• Per-file statistics are parsed, and a commit message is unaffected by the flag (tested)
• The count is populated per the classification rules, with the root README.md and doc/html/
  exclusions covered explicitly (tested)
• Renames and binary files are classified without error (tested)
• Commit gains the field with a migration, and a non-destructive re-import heals rows that
  predate it while leaving their ids alone (tested)
• A destructive re-import discards the grants sourced from the commits it deletes, and the counts
  of the affected members are recalculated (tested)
• Pressing "Update Commits" in the admin leaves no grant pointing at a deleted commit, for
  Documenter or for Code Commits
• Merge commits never count as documentation
• A commit whose author is not linked to a member produces no grant, and produces one once that
  author is claimed
• Documentation is selectable as an automatic source in the commands and the admin, without the
  list being restated anywhere
• A backfill of this source alone awards Documenter, and a second one creates no additional grants
• Nothing describes Documentation as manual-only or as having no automatic source any more
• No additional GitHub API calls: the new data comes from the local clone the importer already
  makes
• Volume is sane when checked against the real commit history, per the check below
• Full suite green, pre-commit clean

Risks & considerations

The numbers have to be checked before this is believed. The thresholds were set without any real
doc-commit data existing, and 50 doc commits for Diamond is a guess. After the first library import
with the field populated, compare the doc-commit count of a few well-known contributors against what
the repository history actually shows, and only then decide whether the ladder needs retuning. That
is a threshold decision and not a change to this iterator.

The same commit is imported once per Library row that shares its repository. Libraries parsed
out of one module's libraries.json all carry that module's repository URL, so a module holding
several libraries has the same sha imported under each of them. This already inflates Code Commits
and will inflate Documenter identically, and it matters more here because the thresholds are much
lower. Worth measuring as part of the volume check above, and fixing separately if it is material.

Re-importing everything is one button press, and that is the problem. "Update Commits" already
re-imports every library, so populating doc data across history needs no new tooling. But it takes the
destructive path, so it is also the single action most likely to orphan grants at scale, and it clones
every library in sequence while it does so. Until it has run, the counts of anyone whose doc work
predates this ticket read low, and a badge that goes up later for no visible reason is a support
question. Sequence it deliberately: land the grant-discarding change first, then run the import.

Grants only appear after a backfill, not when the commits land. The weekly release pipeline already
sweeps every registered source at the end of its run, so this source joins that sweep as soon as it is
registered and needs no pipeline change. The commit import step itself deliberately does not sweep, to
avoid walking the commit table twice per release.

Nothing here removes a grant. If a library's history is rewritten so that a commit no longer touches
documentation, the grant survives until someone reconciles. That split is the engine's design and not
specific to this source.

Open questions

• Count or boolean? Storing the number of doc files touched costs the same as a boolean and keeps
  future weighting possible; a boolean is marginally cheaper to reason about. Recommend the count.
• Does the achievement's description still describe what is being measured? It currently frames
  Documenter around standardised Boost tooling, Antora and BoostLook specifically, while this source
  counts any commit touching a doc path or a doc file extension. Either the description is reworded or
  the classifier is narrowed, and the two should be settled together before the first production
  backfill.
• Does any repository count entirely as documentation? The spike left open whether a docs-only
  repository should be treated as such wholesale rather than path by path. No such repository is in
  scope for this ticket, so the question only needs an answer if one is added.
• Should the grant point at the commit, or at something coarser? Pointing at the commit mirrors
  Code Commits and is the obvious default, but it also means the grant count moves with the
  double-counting described above. Worth confirming before the first backfill rather than after.
• Should the "Update Commits" button stop deleting rows? Its hardcoded destructive mode is what
  creates the orphaning hazard in the first place; passing the non-destructive mode instead would make
  the hazard mostly theoretical, since the importer already updates matched rows in place. It is a
  change to existing commit-import behaviour with its own reasons for being that way, so it is called
  out here rather than absorbed. Discarding grants before the delete is the correct fix either way and
  should land regardless of how this is answered.