Skip to content

feat(hosts): add Grok CLI host adapter - #52

Merged
phodal merged 6 commits into
QoderAI:mainfrom
Kyncc:feat/grok-host-adapter
Aug 3, 2026
Merged

feat(hosts): add Grok CLI host adapter#52
phodal merged 6 commits into
QoderAI:mainfrom
Kyncc:feat/grok-host-adapter

Conversation

@Kyncc

@Kyncc Kyncc commented Aug 2, 2026

Copy link
Copy Markdown

Summary

  • Add a first-class Grok CLI host adapter for configured assets (scripts/agent-customize/providers/grok.mjs) and session evidence (scripts/session-analysis/platforms/grok.mjs).
  • Wire Grok into session-analysis, agent-customize, asset baseline/integrity, evidence-bundle, report-run, task-loop-source, and portable HTML report routing (.grok/better-harness).
  • Document Grok as Adapter support (not Verified Quickstart) in the adapter matrix, homepage cards, README, and sessions diagnostics.
  • Make walkFiles follow directory symlinks so Grok skills installed with ln -s under ~/.grok/skills are inventoried.

Spec / acceptance

See docs/specs/2026-08-02-grok-host-adapter.md (Grok-A1/A2, Grok-S1–S4, Grok-R1/R2).

Install path for local use:

ln -sfn <repo>/skills/better-harness ~/.grok/skills/better-harness
# then /better-harness in Grok TUI

Test plan

  • Focused unit tests: node --test test/session-analysis-providers.test.mjs test/agent-customize.test.mjs test/support-declarations.test.mjs test/coding-agent-platform-notes.test.mjs test/docs-entrypoints.test.mjs test/better-harness-evidence-bundle.test.mjs test/docs-site.test.mjs (126 pass)
  • Local smoke: session-analysis sources --platform grok discovers workspace sessions
  • Local smoke: agent-customize inventory --provider grok finds symlink-installed better-harness skill
  • Local smoke: harness evidence-bundle --platform grok --depth quick returns status=complete with available lanes
  • CI full suite on this PR
  • Maintainer: optional interactive /better-harness report-loop smoke before Quickstart promotion

Register Grok as a first-class adapter for configured assets, session
evidence, evidence bundles, and portable HTML report routing. Skills
installed via symlink under ~/.grok/skills are now inventoried.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

本 PR 在 Better Harness 中新增 Grok CLI 作为一等公民的宿主适配器:补齐会话证据读取(session-analysis)、已配置资产盘点(agent-customize / coding-agent-practices)、证据束(evidence-bundle)与 HTML 报告输出路由(.grok/better-harness),并同步更新文档矩阵与站点入口以将 Grok 标注为 Adapter support(非 Verified Quickstart)。实现与验收标准对齐到 docs/specs/2026-08-02-grok-host-adapter.md(Grok-A1/A2、Grok-S1–S4、Grok-R1/R2)。

Changes:

  • 新增 Grok 会话平台:实现对 ~/.grok/sessions/<url-encoded-cwd>/<session-id>/ 的发现、范围限定、事件归一化与可选 signals.json 用量读取
  • 新增 Grok 资产盘点:读取 config.toml MCP servers、skills、hooks、installed plugins 与 workspace .grok / .agents 资产
  • 更新 CLI/路由/文档与测试:将 grok 纳入平台枚举、帮助文案、适配矩阵、主页卡片与相关单测

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/support-declarations.test.mjs 扩展支持平台列表并更新 portable HTML 路由一致性断言以包含 Grok
test/session-analysis-providers.test.mjs 为 Grok provider 添加 dispatcher 覆盖与会话事件/隔离/缺失 signals 的单测
test/docs-site.test.mjs 更新站点统计与矩阵边界断言(adapter 数从 2→3)
test/docs-entrypoints.test.mjs 将 Grok 加入 adapter-support hosts、主页卡片计数与中英矩阵断言
test/coding-agent-platform-notes.test.mjs 将 Grok 纳入 Sessions Diagnostics 的平台说明与支持列表
test/better-harness-evidence-bundle.test.mjs evidence-bundle help 覆盖 Grok,并检查 --grok-home
test/agent-customize.test.mjs 新增 Grok 资产盘点与 --grok-home 覆盖的端到端测试夹具
templates/reporting/routing.md 将 Grok 加入 portable HTML report 的宿主路由条件
scripts/session-analysis/usage-summary.mjs --platform help 列表加入 grok
scripts/session-analysis/selection-profile.mjs --platform help 列表加入 grok
scripts/session-analysis/platforms/grok.mjs 新增 Grok 会话证据适配器实现(sources/sessions/events/facts)
scripts/session-analysis/lifecycle-demand-signals.mjs safeHost 白名单加入 grok
scripts/session-analysis/fs.mjs walkFiles 改为可跟随目录 symlink(用于 Grok skills 软链场景)
scripts/session-analysis/analyzer.mjs 注册 Grok 平台模块并更新帮助与 --grok-home
scripts/harness-analysis/task-loop-source.mjs task-loop-source 平台枚举与 inventory 路由加入 grokHome/grok
scripts/harness-analysis/report-run.mjs report 平台枚举与允许参数集合加入 grok/grok-home
scripts/harness-analysis/render-report.mjs 为 grok 的 html 输出默认目录引入 .grok/better-harness 分支
scripts/harness-analysis/evidence-bundle/contract.mjs evidence-bundle providers 集合加入 grok
scripts/harness-analysis/evidence-bundle/cli.mjs evidence-bundle CLI 帮助/参数白名单加入 --grok-home
scripts/harness-analysis/evidence-bundle/agent-customize.mjs asset providers 集合加入 grok
scripts/coding-agent-practices/inventory.mjs 平台提示与 userBase 推断加入 grokHome
scripts/coding-agent-practices/asset-integrity.mjs provider 白名单与错误文案加入 grok
scripts/coding-agent-practices/asset-baseline.mjs provider 白名单与错误文案加入 grok
scripts/agent-lint/cli.mjs agent-assets-review provider 列表加入 grok
scripts/agent-customize/providers/index.mjs 注册 Grok provider collector
scripts/agent-customize/providers/grok.mjs 新增 Grok 配置资产盘点(skills/hooks/mcp/plugins/workspace primitives)
scripts/agent-customize/cli.mjs CLI usage/参数透传加入 --grok-home
references/session-evidence/sessions-diagnostics.md 文档增加 Grok 平台诊断与证据位置说明
README.zh-CN.md 文案更新:总适配器数与 adapter-support 列表加入 Grok
README.md 文案更新:总适配器数与 adapter-support 列表加入 Grok
docs/src/pages/index.js 主页新增 Grok host card,并更新架构说明/QuickStart 文案
docs/specs/2026-08-02-grok-host-adapter.md 新增 Grok 适配器 spec(验收、隐私边界、冒烟步骤)
docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/installation.mdx 中文安装页:适配器总数更新为 9
docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/hosts/adapter-matrix.md 中文矩阵:加入 Grok 条目与边界说明
docs/i18n/zh-Hans/code.json 中文主页翻译新增 Grok host card 文案并更新计数文案
docs/docs/installation.mdx 英文安装页:适配器总数更新为 9
docs/docs/hosts/adapter-matrix.md 英文矩阵:加入 Grok 条目与边界说明
docs/adapters/README.md Canonical 矩阵:加入 Grok 行与相关说明段落
assets/better-harness-architecture-en.svg 架构图文字更新:capability adapters 数量与 adapter-support 列表加入 Grok
Suppressed comments (1)

docs/src/pages/index.js:537

  • This QuickStart intro still says "Eight host adapters" even though the rest of the PR bumps the declared total to nine. Update the English copy to match the docs and tests.
            Eight host adapters are supported. Six have verified setup paths;
            Pi, WorkBuddy, and Grok link to their current support boundaries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +82 to 86
// Dirent isDirectory/isFile are false for symlinks; follow them so hosts
// that install skills via ln -s (e.g. Grok ~/.grok/skills) are inventoried.
const linkedDirectory = entry.isSymbolicLink() ? await isDirectory(fullPath) : false;
if (entry.isDirectory() || linkedDirectory) {
await visit(fullPath, depth + 1);
Comment thread docs/src/pages/index.js Outdated
Comment on lines 512 to 514
Eight capability-level host adapters feed the same evidence
pipeline. Six have verified Quickstart paths; Pi and WorkBuddy keep
pipeline. Six have verified Quickstart paths; Pi, WorkBuddy, and Grok keep
their current adapter-support boundaries explicit.
Comment on lines +90 to +94
options.out ??= options.mode === "cursor-canvas"
? ".cursor/better-harness"
: options.mode === "html" && String(options.platform ?? options.provider ?? "").toLowerCase() === "grok"
? ".grok/better-harness"
: ".qoder/better-harness";
Update frozen help fixtures, CLI registry summary, portable-HTML routing
assertions, and Windows-portable Grok session path encoding checks.
OnMyAgent keeps SoT under .agents/skills and links .grok/skills to it.
Inventory previously listed both paths, inflating owner routes past the
16-entry baseline and marking agentCustomize partial for normal depth.
@phodal

phodal commented Aug 2, 2026

Copy link
Copy Markdown
Member

Local validation update

I completed local validation against PR head fe4739f using Grok CLI 0.2.118 and Node.js 22.20.0.

The isolated skill-symlink path works: both native grok inspect --json and the Better Harness adapter discover better-harness when it is linked under $GROK_HOME/skills. The 127 focused tests pass, the full Node test suite exits successfully, pack:verify passes, and the English/Chinese Docusaurus production build succeeds. The current GitHub CI jobs are also green.

However, local validation found several issues that I believe should be addressed before merge:

  1. The configured-asset inventory does not match Grok's effective configuration. In the same workspace, native grok inspect --json reported 5 plugins, 28 skills, 1 hook, and 5 MCP servers, while agent-customize inventory --provider grok reported 0, 25, 0, and 0. The adapter hard-codes $GROK_HOME/installed-plugins, but Grok 0.2.118 discovers plugins from ~/.grok/plugins, project .grok/plugins, configured plugin paths, and compatibility sources. It also misses user ~/.agents/skills, project .grok/config.toml, and effective enablement/precedence.

  2. Tool lifecycle events are inflated. The adapter treats every tool_call_update as a terminal result, including progress and status-less updates. In one real local session, 20 raw tool calls became 46 normalized tool.result events, and all 20 invocation IDs had duplicate results.

  3. Token usage is mapped incorrectly. signals.contextTokensUsed is treated as total token consumption, while the actual _x.ai/session/update:turn_completed usage records are reduced to metadata. In the sampled session, the adapter emitted 44,532 total tokens while the recorded turn usage summed to 279,425.

  4. Messages are double-counted. Grok documents updates.jsonl as the authoritative conversation log, but the adapter also converts chat_history.jsonl into user/assistant events without correlation or deduplication.

  5. The Grok HTML default output route is unreachable. render-report.mjs checks options.platform/options.provider, but its argument parser rejects both flags. Running with --platform grok returns Unknown argument: --platform.

  6. Long workspace paths are not discoverable. When an encoded cwd exceeds 255 bytes, Grok uses a slug-plus-hash group directory and stores the original path in .cwd. The adapter only matches the direct encodeURIComponent(cwd) name.

  7. Directory-symlink traversal needs realpath-based visited tracking. Existing callers are depth-bounded, so the current automated review overstates the immediate unbounded-recursion risk, but cycles can still duplicate traversal and the shared helper defaults to infinite depth.

  8. A few readiness details remain open. The English homepage still says “Eight” adapters in two places; git diff --check fails on four trailing-whitespace lines in the spec; the spec smoke command invokes scripts/session-analysis/analyzer.mjs, which exits successfully without producing output instead of using the public scripts/session-analysis.mjs entrypoint; and the three commits do not include the repository-required single Co-authored-by line.

Because the synthetic fixtures currently encode several assumptions that differ from Grok 0.2.118's native behavior, green tests alone do not validate the claimed “Verified assets + sessions + HTML render path” boundary. Please align the adapter and fixtures with the native CLI output, then repeat the real-host comparison.

Address PR QoderAI#52 review feedback from maintainer validation:

- Inventory plugins from ~/.grok/plugins, project .grok/plugins, and
  config paths; load project .grok/config.toml MCP; include ~/.agents/skills;
  avoid double-counting when workspace is the user home.
- Treat only terminal tool_call_update statuses as results; map
  turn_completed.usage for token spend; never use contextTokensUsed as total.
- Prefer updates.jsonl over chat_history; discover long-path session groups
  via .cwd markers.
- Accept --platform/--provider on harness render for .grok/better-harness;
  cycle-safe walkFiles; homepage Nine copy; spec smoke path cleanup.

Co-authored-by: Grok (Grok 4.5) <grok@x.ai>
@Kyncc

Kyncc commented Aug 3, 2026

Copy link
Copy Markdown
Author

Follow-up: addressed local validation findings

Pushed f0062db on this branch in response to the review.

# Finding Fix
1 Inventory vs grok inspect Plugins from ~/.grok/plugins, project .grok/plugins, config plugins.paths; project .grok/config.toml MCP; ~/.agents/skills; avoid double-count when workspace is $HOME
2 Inflated tool results Only terminal tool_call_update statuses (completed/failed/…) emit tool.result; progress/null → metadata.tool_call_update
3 Token usage Prefer _x.ai/session/update turn_completed.usage; never map signals.contextTokensUsed as totalTokens (context window stays metadata)
4 Double-counted messages updates.jsonl authoritative; chat_history.jsonl only when updates missing
5 HTML default out unreachable harness render accepts --platform/--provider; html defaults to .<platform>/better-harness
6 Long cwd groups Discover slug+hash groups via .cwd marker in addition to encodeURIComponent names
7 Symlink cycles walkFiles tracks realpath of visited dirs
8 Readiness polish Homepage “Nine…”, spec smoke uses session-analysis.mjs, trailing whitespace cleaned, commit includes single Co-authored-by

Tests: Grok session + agent-customize suites green locally. Please re-run your real-host comparison against this head when convenient.

@phodal

phodal commented Aug 3, 2026

Copy link
Copy Markdown
Member

Follow-up review of f0062db

Reviewed the follow-up commit only, in a clean worktree at PR head f0062db, Node.js 22.20.0.

Verification status first, so the evidence boundary is explicit:

  • Full npm test on PR head: 1101 pass / 1 fail. The single failure is better-harness CLI short-circuits help for every registered terminal path (10s per-spawn timeout). The same test also fails on main in this environment (different leaf path), and running the file alone on PR head passes 40/40. I treat it as a pre-existing flake under parallel load, not a regression from this PR.
  • The real-host comparison you asked for could not be repeated. This machine currently has no ~/.grok and no grok binary, so findings 1-4 are still backed only by synthetic fixtures. My checks below are code reads plus targeted probes against the adapter modules.
  • Unrelated to this PR: main currently fails the doc-link test (docs/specs/2026-08-02-contained-structural-symlinks.md -> ../AGENTS.md). This branch predates it and will meet it on rebase.

Blocking

1. docs/specs/2026-08-02-grok-host-adapter.md is corrupted by an editing script.

Lines 36-47 contain leftover Python patch code that was committed as spec prose:

'
    )
# add notes section if missing
if 'updates.jsonl is authoritative' not in t:
    t = t.replace(
        '### Privacy
',

Consequences: the ### Workspace qualification body is gone, ### Privacy now appears twice, and the four new evidence notes live inside the broken block. Nothing in npm test lints spec bodies, so this would land on main as-is. Please rewrite the section by hand.

Related: the Native contract table still declares Plugins as $GROK_HOME/installed-plugins/ and marketplace-cache/, which contradicts the new multi-root discovery in this commit.

2. Plugin inventory still inflates counts (finding 1 not fully closed).

collectGrokPlugins dedupes on grok/${installMatch}/${name}, so one physical plugin directory reached through several roots is counted once per root. Probe result with a single plugin directory:

A plugins: 3 [
  'grok/grok-plugins-dir/sample-plugin',
  'grok/grok-project-plugins-dir/sample-plugin',
  'grok/grok-plugins-path-config/sample-plugin'
]

Two realistic triggers:

  • [plugins] paths lists a directory that is already a default root.
  • workspace is the user home, so workspace/.grok/plugins is $GROK_HOME/plugins. This is exactly the double-count case the commit guards for skills, MCP, and rules via projectIsUserHome, but plugins skip that guard.

Suggested fix: key the map on realpath(pluginRoot) and merge the discovered roots into installSources instead of emitting separate records.

3. turn_completed.usage still drops nested usage (finding 3 not fully closed).

normalizeUsageFromTurn reads only flat inputTokens / outputTokens / totalTokens / cachedReadTokens, while the same block reads update.usage.modelUsage to derive the model name — so the code already assumes the nested shape exists. With nested usage the event degrades to metadata and token spend stays unobserved:

// updates.jsonl record
{ method: "_x.ai/session/update", params: { update: { sessionUpdate: "turn_completed",
  usage: { modelUsage: { "grok-4": { inputTokens: 1000, outputTokens: 200, totalTokens: 1200 } } } } } }

// normalized events
"eventCounts": { "metadata.turn_completed": 1 }   // no model.response.completed, no modelUsage

Either aggregate the modelUsage sub-objects, or record real turn_completed samples proving the flat fields are always present. Given that the previous round measured 279,425 recorded turn tokens on a real session, this needs host evidence rather than a fixture.

Non-blocking

4. includeUserHome: false leaks a user-home path. diagnostics.installedPluginRecordFiles is now taken from the unfiltered scan, so a project-scope run reports ~/.grok/plugins while plugins is empty and installedPluginState says missing. That gate existed before this commit, and scripts/coding-agent-practices/asset-baseline.mjs calls the provider with includeUserHome: false.

5. Plugin enabled is a heuristic, and precedence is still unverified. The comment says plugins are off by default unless listed in enabled, but the code defaults to true when the config declares neither list. Only the user config.toml [plugins] table is read; project .grok/config.toml is not. Since finding 1 explicitly mentioned effective enablement and precedence, this should either be backed by native output or documented as a stated approximation in the spec.

6. harness render --platform has no allowlist. Any value now produces .<value>/better-harness in html mode, so --platform grock silently writes .grock/better-harness instead of failing. The generalization itself matches the .<provider>/better-harness routing in SKILL.md, but the value should be validated against the supported host list.

7. Terminal tool status set is narrow. completed/failed/error/cancelled/canceled matches ACP, but if Grok ever emits success or done, every tool result silently becomes metadata and tool success rate collapses to zero. A test or a warning for unrecognized terminal-looking statuses would make the failure visible.

8. Co-author trailer is only on this commit. 537a2e5, 9d6db9c, and fe4739f still carry none, so a non-squash merge would not satisfy the repository requirement.

Rewrite the corrupted host-adapter spec, dedupe plugins by realpath while
merging installSources, aggregate nested turn_completed.usage.modelUsage,
validate harness render --platform against supported hosts, and keep
includeUserHome=false diagnostics free of user-home plugin paths.

Co-authored-by: Grok (Grok 4.5) <grok@x.ai>
@Kyncc

Kyncc commented Aug 3, 2026

Copy link
Copy Markdown
Author

Follow-up: closed blocking items from the f0062db review

Pushed $(git rev-parse --short HEAD) addressing the three blocking findings:

  1. Spec corruption — rewrote docs/specs/2026-08-02-grok-host-adapter.md by hand (workspace qualification, conversation evidence, privacy, stated approximations). Native contract table now documents multi-root plugins + long-path .cwd groups.
  2. Plugin inflation — inventory keys physical plugins by realpath(pluginRoot), merges discovery roots into installSources, and skips treating workspace/.grok/plugins as a separate root when it is the user home tree. includeUserHome: false no longer records user-home plugin paths in diagnostics.
  3. Nested usagenormalizeUsageFromTurn aggregates usage.modelUsage.<modelId> when flat totals are absent (and fills missing totals from nested objects).

Also closed related non-blocking items:

  • harness render --platform fail-closed allowlist for supported hosts
  • Documented plugin enablement approximation in the spec / unsupported list

Local: Grok session + agent-customize suites green; plugin multi-root probe returns count 1.

Merge user and project [plugins] tables for enable lists, and skip platform
validation when render --help is requested.

Co-authored-by: Grok (Grok 4.5) <grok@x.ai>
@phodal
phodal merged commit 72ad82d into QoderAI:main Aug 3, 2026
4 checks passed
@phodal

phodal commented Aug 3, 2026

Copy link
Copy Markdown
Member

Merged as 72ad82d. Thanks for the Grok adapter, and for turning around three review rounds so quickly — the spec rewrite, the realpath plugin dedupe, and the nested turn_completed.usage handling all check out against my local probes, and CI was green on all four matrix jobs.

I did not want to send you into a fourth round for the last two items, so I picked them up myself in #55:

  • The realpath dedupe also rewrote every plugin id to grok/plugin/<dirname>, so a user-scope and a project-scope plugin sharing a directory name collapsed onto one id. asset-integrity groups name collisions and then dedupes by id, so the plugin-name-collision finding went silent exactly where it matters. Ids are now discovery-root qualified again, with the dedupe map still keyed on realpath.
  • The user and project [plugins] tables were concatenated into one text before parsing, so a project-declared paths replaced the user-declared one. Each config is parsed separately now and the lists are unioned.

Also included there: nested modelUsage fills any field a partial flat usage record omitted, diagnostics.configPath has a single meaning again, and the render platform allowlist is pinned to the session platform registry with tests for the .grok/better-harness default route and fail-closed unknown platforms.

One note for next time: the real-host comparison could not be repeated on my side (no ~/.grok or grok binary available in that environment), so the native-parity parts of this adapter are still fixture-backed. If you keep using Grok CLI day to day, a follow-up PR that records a real grok inspect --json versus agent-customize inventory --provider grok comparison would let us promote Grok from adapter support to a verified Quickstart path.

creayma-del pushed a commit to creayma-del/better-harness that referenced this pull request Aug 3, 2026
…ess-gate

Integrate main (PRs QoderAI#51-QoderAI#55) into the loop-readiness feature branch.
Resolved conflict in test/scripts-refactor-contract.test.mjs:
- Recalculated sha256 baselines for `commands --json` and `schema` to
  reflect the merged command registry (Grok host adapter added by PR QoderAI#52).

All 1120 tests pass after merge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants