chore: fix ordering and a typo - #371
Merged
Merged
Conversation
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
Contributor
|
Claude finished @mbevc1's task in 1m 54s —— View job Doc review — PR #371
No Critical findings. Two Suggestions, both posted inline. Nothing blocks merge.
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| ## New features | ||
|
|
||
| - **Custom attestation type summaries** — `kosli create attestation-type` and `kosli update attestation-type` now accept `--summary-json`, a JSON list of ordered `{name, expression}` entries. Each expression is a jq expression evaluated against the attestation payload, and Kosli renders the results as labeled rows on the attestation detail page. See [custom attestation types](/getting_started/attestations). | ||
| - **Custom attestation type summaries** — `kosli create attestation-type` now accepts `--summary`, a repeatable `'NAME=EXPRESSION'` entry (for example `--summary "Critical=.critical_count"`), and `--summary-json`, the same list given as a JSON array of `{name, expression}` entries. Each expression is a jq expression evaluated against the attestation payload, and Kosli renders the results as labeled rows on the attestation detail page, in the order given. The two flags cannot be combined. See [custom attestation types](/getting_started/attestations) and the [`kosli create attestation-type` reference](/client_reference/kosli_create_attestation-type). |
Contributor
There was a problem hiding this comment.
Suggestion — the rewrite is accurate: it matches client_reference/kosli_create_attestation-type.md:28-50 (repeatable --summary, JSON --summary-json, "The two summary flags cannot be combined"), and dropping kosli update attestation-type is a real correctness fix — that command doesn't exist, kosli create attestation-type is documented as "Create or update".
One loose end: the first link, /getting_started/attestations, doesn't mention summaries anywhere on the page (no summary match in getting_started/attestations.md). A reader following it for summary docs lands on nothing. Either add a short summaries section to that page or drop the link and keep only the reference link, which does cover it.
|
|
||
| </Update> | ||
|
|
||
| <Update label="August 18, 2026" description="v2.37.0" tags={["CLI"]}> |
Contributor
There was a problem hiding this comment.
Suggestion — the date ordering fix is correct; the whole file is now strictly descending by label.
Within August 18 the order is now Platform → CLI → Terraform. On every other shared date in this file the Platform entry comes last: Aug 19 (CLI, Platform), Aug 14 (CLI, Platform), Jul 31 (CLI, Platform), Jul 28 (CLI, CLI, Platform), Jul 10 (Terraform, CLI, Platform), Jul 8 (CLI, Terraform, Platform). Moving this v2.37.0 block above the Aug 18 Platform entry (currently line 40) would match that convention.
Non-blocking — it only affects same-day grouping, not the date ordering this PR set out to fix.
dangrondahl
added a commit
that referenced
this pull request
Sep 3, 2026
Reviewed the last 20 non-bot PRs (14 touched docs) against what `doc-write` and `doc-review` actually ask for, then fixed what the record showed was wrong. No site content changes — this is all agent tooling. ## Why `doc-review` produced **0 Critical, 5 Improvement and ~20 Suggestion** findings across those 14 PRs. Its genuinely valuable catches were all cross-file consistency checks the skill never asked for: - a changelog entry documenting `kosli update attestation-type`, a command with no reference page (#371) - `template-reference/flow_template.md` out of sync with the schema the same PR regenerated (#296) - the one file an approvals-removal sweep missed, `understand_kosli/how_kosli_works.md:22` (#345) Its weakest findings were prose polish, some already fixed at branch head. Meanwhile its written checklist was dominated by things that always pass or are already enforced by `vale-spellcheck`. Two structural gaps the PR record made obvious: - **Placement was never questioned.** In #305 a pure reference page was authored into `integrations/`, and a human reviewer had to ask for the move — commit `9e7a121 docs: move GitHub Action reference into the Reference section`. That question should come from the review. - **Generated pages were handled inconsistently.** #375 got it right and said so. #374 wrote *"the durable fix is in the `kosli-dev/cli` generator — a hand-edit here is overwritten by the next release"* and then emitted a `Fix this` link scoped to `repo=kosli-dev/docs` telling an agent to edit the generated file anyway, plus 4 inline comments on regenerated files. ## What changed **`doc-review`** — promoted the three accidental wins to named checks, each carrying its precedent. Added placement, redirects and anchor stability. Added an explicit *what not to report* bar and an 8-finding cap. Stopped hand-checking spelling. Dropped the "what looks good" recital that the sticky comment re-renders on every push. **Generated pages** — split into three categories rather than one blanket ban: deterministically regenerated (edit is deleted), agent-synced (edit survives but drifts), and hand-authored despite the directory (`client_reference/overview.md`, `output_and_verbosity.md`). Includes the filename→source mapping, verified upstream: `kosli_attest_sonar.md` ← `cmd/kosli/attestSonar.go`. Also records that **`^` is the CLI's backtick convention** in Go long descriptions, substituted by `kosli docs`. That makes the `^jq^` defect #374 found a generator escaping bug in the Accordion-title path, not a typo in the Go string — so a reviewer can point at the right fix. Worth filing upstream separately. **`doc-write`** — added a Diátaxis→tab placement table so #305 can't recur, plus redirects, anchor stability, and the generated-paths table. **New `doc-structure` skill + monthly workflow** — audits navigation shape and changelog coverage, which per-PR review structurally cannot see, and files issues. Read-only against docs; issues are its only write. Capped at 8 issues, deduplicating against open issues first — its headline check reproduces the `/getting_started/attestations` summary gap, which is already open as #364. **New `scripts/audit_navigation.py` + 22 tests** — the audit's mechanical checks, extracted from an inline heredoc. Three payoffs: 1. `pr-quality.yml` already runs `pytest tests/`, so **CLAUDE.md core rule 2 is now enforced deterministically** — a page file with no `navigation` entry fails the build. No new job needed. 2. `doc-structure.yml` can allow `Bash(python3 scripts/audit_navigation.py:*)` instead of `Bash(python3:*)`, which was arbitrary code execution. 3. Determinism. The inline version had already shipped a bug: `sed 's|\.mdx\?$||'` is a no-op on BSD sed, so every page looked orphaned on macOS. That case is now a regression test. Integrity findings (orphans, dangling entries) are separated from shape findings (single-child groups, deep nesting, Title Case labels, oversized groups, inconsistent icons). Only integrity can fail a build — the script cannot tell a group that should be merged from one deliberately kept separate. `Reference ▸ CLI Reference` is exempt from shape checks: `update-cli-nav.py` generates it from the CLI's command tree, so a single-child `kosli allow` group is upstream truth. Without that exemption the audit reported 54 findings, 21 of them proposing to reshape generated navigation — the same mistake being fixed in `doc-review`. It now reports 33, all hand-maintained. **New `docs-restructure` issue template** — Markdown, not a YAML form: `gh issue create --body` doesn't apply templates mechanically and YAML forms can't be filled from the CLI, so a form would help humans and do nothing for the job. Its URL-impact section encodes the distinction that decides whether a restructure is safe — a group rename changes no URLs, a moved page file needs a `config/redirects.json` entry. `config.yml` keeps blank issues enabled. **`CLAUDE.md`** — documented the six automated PR checks, the generated-page source map, the audit script, and the three skills. Removed the pointer to a `changelog-creator` skill that doesn't exist in this repo. ## Anti-rot pass Swept all three skills for dated claims. Counts ("three of the last fourteen PRs") and current-state assertions ("link-rot reports skipping on most PRs") became durable rules. The link-rot guidance now routes through `gh pr checks` so it self-heals if Mintlify starts running it reliably. Past-tense `Precedent:` items were kept — they're what make the checks concrete. ## Verification - `python3 -m pytest tests/` — 44 passed (22 pre-existing + 22 new) - `python3 scripts/audit_navigation.py --check` — exit 0, integrity clean - `mint broken-links` — no new broken links - Both workflows parse; triggers and permissions confirmed ## Follow-ups, not in this PR - `tutorials/working_with_controls.mdx:24` links to `/getting_started/service-accounts`, which has never existed — the page is `/administration/authentication/service_accounts`. Broken on `main` since #154; `link-rot` reports `skipping`, which is why it went unseen. - The `mintlify-docs` plugin in `kosli-plugins` still ships near-duplicate `doc-writer`/`doc-reviewer` agents that say "update `docs.json` navigation" — the pre-`config/` layout. Needs its own PR there. - The reviews twice asked for `python3` in `doc-review.yml`'s `--allowedTools` to validate JSON payloads and fence balance. Left alone — `Bash(python3:*)` is a security-surface call worth making deliberately.
gsavage
pushed a commit
that referenced
this pull request
Sep 7, 2026
Reviewed the last 20 non-bot PRs (14 touched docs) against what `doc-write` and `doc-review` actually ask for, then fixed what the record showed was wrong. No site content changes — this is all agent tooling. ## Why `doc-review` produced **0 Critical, 5 Improvement and ~20 Suggestion** findings across those 14 PRs. Its genuinely valuable catches were all cross-file consistency checks the skill never asked for: - a changelog entry documenting `kosli update attestation-type`, a command with no reference page (#371) - `template-reference/flow_template.md` out of sync with the schema the same PR regenerated (#296) - the one file an approvals-removal sweep missed, `understand_kosli/how_kosli_works.md:22` (#345) Its weakest findings were prose polish, some already fixed at branch head. Meanwhile its written checklist was dominated by things that always pass or are already enforced by `vale-spellcheck`. Two structural gaps the PR record made obvious: - **Placement was never questioned.** In #305 a pure reference page was authored into `integrations/`, and a human reviewer had to ask for the move — commit `9e7a121 docs: move GitHub Action reference into the Reference section`. That question should come from the review. - **Generated pages were handled inconsistently.** #375 got it right and said so. #374 wrote *"the durable fix is in the `kosli-dev/cli` generator — a hand-edit here is overwritten by the next release"* and then emitted a `Fix this` link scoped to `repo=kosli-dev/docs` telling an agent to edit the generated file anyway, plus 4 inline comments on regenerated files. ## What changed **`doc-review`** — promoted the three accidental wins to named checks, each carrying its precedent. Added placement, redirects and anchor stability. Added an explicit *what not to report* bar and an 8-finding cap. Stopped hand-checking spelling. Dropped the "what looks good" recital that the sticky comment re-renders on every push. **Generated pages** — split into three categories rather than one blanket ban: deterministically regenerated (edit is deleted), agent-synced (edit survives but drifts), and hand-authored despite the directory (`client_reference/overview.md`, `output_and_verbosity.md`). Includes the filename→source mapping, verified upstream: `kosli_attest_sonar.md` ← `cmd/kosli/attestSonar.go`. Also records that **`^` is the CLI's backtick convention** in Go long descriptions, substituted by `kosli docs`. That makes the `^jq^` defect #374 found a generator escaping bug in the Accordion-title path, not a typo in the Go string — so a reviewer can point at the right fix. Worth filing upstream separately. **`doc-write`** — added a Diátaxis→tab placement table so #305 can't recur, plus redirects, anchor stability, and the generated-paths table. **New `doc-structure` skill + monthly workflow** — audits navigation shape and changelog coverage, which per-PR review structurally cannot see, and files issues. Read-only against docs; issues are its only write. Capped at 8 issues, deduplicating against open issues first — its headline check reproduces the `/getting_started/attestations` summary gap, which is already open as #364. **New `scripts/audit_navigation.py` + 22 tests** — the audit's mechanical checks, extracted from an inline heredoc. Three payoffs: 1. `pr-quality.yml` already runs `pytest tests/`, so **CLAUDE.md core rule 2 is now enforced deterministically** — a page file with no `navigation` entry fails the build. No new job needed. 2. `doc-structure.yml` can allow `Bash(python3 scripts/audit_navigation.py:*)` instead of `Bash(python3:*)`, which was arbitrary code execution. 3. Determinism. The inline version had already shipped a bug: `sed 's|\.mdx\?$||'` is a no-op on BSD sed, so every page looked orphaned on macOS. That case is now a regression test. Integrity findings (orphans, dangling entries) are separated from shape findings (single-child groups, deep nesting, Title Case labels, oversized groups, inconsistent icons). Only integrity can fail a build — the script cannot tell a group that should be merged from one deliberately kept separate. `Reference ▸ CLI Reference` is exempt from shape checks: `update-cli-nav.py` generates it from the CLI's command tree, so a single-child `kosli allow` group is upstream truth. Without that exemption the audit reported 54 findings, 21 of them proposing to reshape generated navigation — the same mistake being fixed in `doc-review`. It now reports 33, all hand-maintained. **New `docs-restructure` issue template** — Markdown, not a YAML form: `gh issue create --body` doesn't apply templates mechanically and YAML forms can't be filled from the CLI, so a form would help humans and do nothing for the job. Its URL-impact section encodes the distinction that decides whether a restructure is safe — a group rename changes no URLs, a moved page file needs a `config/redirects.json` entry. `config.yml` keeps blank issues enabled. **`CLAUDE.md`** — documented the six automated PR checks, the generated-page source map, the audit script, and the three skills. Removed the pointer to a `changelog-creator` skill that doesn't exist in this repo. ## Anti-rot pass Swept all three skills for dated claims. Counts ("three of the last fourteen PRs") and current-state assertions ("link-rot reports skipping on most PRs") became durable rules. The link-rot guidance now routes through `gh pr checks` so it self-heals if Mintlify starts running it reliably. Past-tense `Precedent:` items were kept — they're what make the checks concrete. ## Verification - `python3 -m pytest tests/` — 44 passed (22 pre-existing + 22 new) - `python3 scripts/audit_navigation.py --check` — exit 0, integrity clean - `mint broken-links` — no new broken links - Both workflows parse; triggers and permissions confirmed ## Follow-ups, not in this PR - `tutorials/working_with_controls.mdx:24` links to `/getting_started/service-accounts`, which has never existed — the page is `/administration/authentication/service_accounts`. Broken on `main` since #154; `link-rot` reports `skipping`, which is why it went unseen. - The `mintlify-docs` plugin in `kosli-plugins` still ships near-duplicate `doc-writer`/`doc-reviewer` agents that say "update `docs.json` navigation" — the pre-`config/` layout. Needs its own PR there. - The reviews twice asked for `python3` in `doc-review.yml`'s `--allowedTools` to validate JSON payloads and fence balance. Left alone — `Bash(python3:*)` is a security-surface call worth making deliberately.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Minor fixes following #370