diff --git a/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md b/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md new file mode 100644 index 00000000..1afdfe96 --- /dev/null +++ b/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md @@ -0,0 +1,697 @@ +# Full Port Plan: grok-eco → hawk-eco (Go reimplementation) + +**Status:** Master long-term plan +**Date:** 2026-07-16 +**Meaning of “port”:** Reimplement **every Grok Build capability** in idiomatic **Go** across hawk-eco repos. +**Not meaning:** Copy Rust crates, depend on Grok binaries, or collapse hawk into a Rust monorepo. + +**Source tree:** `grok-eco/grok-build` (~1.35M LOC Rust, 1 product monorepo) +**Target tree:** `hawk-eco/*` (multi-repo Go platform + cloud/TS/Python) + +**Related shorter plan:** `GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md` (control-plane focus). +**This document:** **complete** crate/tool/doc/slash inventory, including small items, with owner repo + status + phase. + +--- + +## 0. Port principles + +1. **Behavior parity over code structure** — same user-visible contracts, not same file tree. +2. **Go multi-repo stays** — map Grok crates into hawk / engines / contracts / cloud / skills. +3. **Prefer wire-first** — Hawk already has partial types; complete wiring before greenfield. +4. **Privacy-first defaults** — Grok Mixpanel/Sentry defaults become **opt-in** OTEL/privacy-safe telemetry. +5. **Multi-provider stays** — Grok sampler/auth maps to **eyrie**, not a single-vendor clone. +6. **Memory stays yaad** — Grok markdown memory maps to **yaad APIs + UX**, not a second store. +7. **Tokens stay tok** — Grok `xai-token-estimation` is obsolete relative to tok. +8. **Track everything** — even 70-line crates appear in the matrix (Done / Partial / Port / Skip / N/A). + +### Status legend + +| Status | Meaning | +|--------|---------| +| **Done** | Hawk-eco already has equivalent or better | +| **Partial** | Exists but incomplete vs Grok | +| **Port** | Must reimplement in Go | +| **Skip** | Intentionally not porting (vendor/stack conflict) | +| **N/A** | Grok internal leaf; absorbed into larger port | + +### Effort bands (one senior Go eng) + +| Band | Meaning | +|------|---------| +| XS | < 1 week | +| S | 1–2 weeks | +| M | 3–6 weeks | +| L | 2–3 months | +| XL | 3–6 months | +| XXL | 6–12+ months (TUI/parity programs) | + +### Suggested calendar (full port) + +| Horizon | Scope | +|---------|--------| +| **Y0 (0–6 mo)** | Control plane + trust + spawn + hooks (critical path) | +| **Y0–1 (6–12 mo)** | Plugins/marketplace, tasks/monitor, ACP, user-guide | +| **Y1–2** | Workspace depth, foreign sessions, hunks, mermaid, PTY, enterprise policy | +| **Y2–3** | Computer-hub class remote tools (if product needs), full slash parity, IDE polish | +| **Y3–5** | Hardening, multi-editor, fleet/agents at scale, continuous Grok-diff audits | + +Full “every small thing green” is a **multi-year product program**, not a 10-year rewrite of Rust. +Critical path remains **months**; **complete parity** including TUI polish and enterprise is **~2–4 years** of continuous work depending on headcount. + +--- + +## 1. Repo mapping (where Grok monorepo goes) + +```text +grok-eco/grok-build (one Rust workspace) + │ + ├─► hawk product CLI/TUI/engine/tools/hooks/plugins/ACP + ├─► hawk-core-contracts pure DTOs (tool/spawn/hooks/policy) + ├─► eyrie LLM routing/stream/retry/catalog/auth credentials + ├─► yaad memory graph + dream UX + ├─► tok compression / secrets / token cost + ├─► trace session capture/import/replay + ├─► sight / inspect review & live audit (no Grok 1:1) + ├─► hawk-mcpkit MCP server helpers + ├─► hawk-community-skills skills + marketplace content + ├─► hawk-cloud managed policy / tenancy / usage + ├─► hawk-sdk-go/python ACP/daemon clients + └─► graycode-core web dashboard only (optional) +``` + +**There is only one Grok product repo (`grok-build`).** +“Port all grok-eco repos” = port **all crates/capabilities inside grok-build** into the **appropriate hawk-eco repos**. + +--- + +## 2. Master crate → hawk-eco matrix + +### 2.1 Product / TUI / shell (codegen) + +| Grok crate | ~LOC | Capability | Target repo | Status | Effort | Notes | +|------------|-----:|------------|-------------|--------|--------|-------| +| `xai-grok-pager` | 415k | Full TUI | hawk `cmd/` | Partial | XXL | Bubble Tea already; parity of panes/modals/slash is multi-year | +| `xai-grok-pager-render` | 35k | Render pipeline | hawk | Partial | L | Streaming render, blocks, media | +| `xai-grok-pager-minimal` | 5.6k | `--minimal` native scrollback | hawk | Partial | M | `hawk --repl` / print modes exist | +| `xai-grok-pager-bin` | 2.9k | Binary composition root | hawk `cmd/hawk` | Done | — | | +| `xai-grok-pager-pty-harness` | 10k | PTY test harness | hawk tests | Partial | M | Test infra | +| `xai-grok-shell` | 336k | Agent runtime host | hawk `internal/engine` | Partial | XXL | Largest runtime port | +| `xai-grok-shell-base` | 2.7k | Shared shell foundation | hawk | Partial | S | Absorb | +| `xai-grok-shell-session-support` | 1.5k | Session support extract | hawk `session` | Partial | S | Absorb | +| `xai-grok-tools` | 112k | Tool implementations | hawk `internal/tool` | Partial | XL | See tool matrix §3 | +| `xai-grok-tools-api` | 0.7k | Tool API / slash wording | hawk + contracts | Partial | S | | +| `xai-grok-workspace` | 77k | FS/VCS/permissions/hub | hawk + sandbox + session | Partial | XL | | +| `xai-grok-workspace-types` | 9.2k | Workspace wire types | contracts | Partial | M | | +| `xai-grok-workspace-client` | 0.8k | Workspace RPC client | hawk (if hub) | Port | M | Only if computer-hub ported | +| `xai-grok-agent` | 21k | Agent defs + system prompt | hawk agent/personas | Partial | L | | +| `xai-grok-subagent-resolution` | 2.6k | Capability/isolation resolve | hawk + contracts | Port | M | Critical | +| `xai-chat-state` | 13k | Chat state actor | hawk session/engine | Partial | L | | +| `xai-grok-markdown` | 22k | Streaming MD TUI | hawk markdown | Partial | L | | +| `xai-grok-markdown-core` | 1.1k | Headless MD analysis | hawk | Partial | S | | +| `xai-ratatui-textarea` | 12k | Input widget | hawk (Bubble Tea) | N/A | — | Different UI stack | +| `xai-ratatui-inline` | 3.7k | Inline render | hawk | N/A | — | UI stack | +| `xai-fast-worktree` | 19k | CoW worktree speed | hawk worktree | Port | L | Perf enhancement | +| `xai-file-utils` | 15k | Event tracking / upload | hawk observability | Partial | M | Privacy opt-in | +| `xai-grok-telemetry` | 14k | Events + OTEL + Mixpanel + Sentry | hawk observability | Partial | L | **Skip Mixpanel default**; keep OTEL | +| `xai-hunk-tracker` | 13k | Agent vs external hunks | hawk | Port | L | | +| `xai-grok-sampling-types` | 13k | Chat API types | **eyrie** | Done* | — | Different shape; eyrie owns | +| `xai-grok-sampler` | 11k | HTTP stream + retry | **eyrie** | Done* | — | Do not replace eyrie | +| `xai-grok-update` | 11k | Auto-update | hawk | Partial | M | | +| `xai-grok-mcp` | 10k | MCP client (oauth, wire) | hawk `mcp` | Partial | L | | +| `xai-codebase-graph` | 9.7k | Tree-sitter graph | hawk codegraph/repomap | Partial | L | | +| `xai-grok-memory` | 9.7k | Cross-session memory | **yaad** | Done* | M | Port UX only | +| `xai-grok-hooks` | 8.3k | File/HTTP hooks | hawk hooks | Port | L | | +| `xai-fsnotify` | 6.7k | FS events | hawk (fsnotify) | Partial | S | | +| `xai-grok-config` | 6k | Config layers + managed | hawk config + cloud | Port | L | | +| `xai-grok-config-types` | 2.7k | Config DTOs | contracts/config | Partial | S | | +| `xai-grok-plugin-marketplace` | 5.3k | Marketplace | hawk + community-skills | Port | L | | +| `xai-grok-shared` | 5.2k | Shared utils | hawk | N/A | — | Absorb | +| `xai-grok-test-support` | 4.6k | Test harness | hawk testutil | Partial | M | | +| `xai-grok-sandbox` | 3.9k | OS sandbox profiles | hawk sandbox | Port | L | | +| `xai-grok-voice` | 2.7k | Streaming STT | hawk | Partial | M | whisper path exists | +| `xai-acp-lib` | 2.3k | ACP protocol | hawk acp | Port | L | | +| `xai-grok-mermaid` | 2.2k | Mermaid→PNG | hawk | Port | M | | +| `xai-crash-handler` | 1.9k | Crash + startup detect | hawk | Port | S | | +| `ptyctl` | 2.3k | Headless PTY control | hawk | Port | M | | +| `ptyctl-cli` | 0.8k | PTY CLI | hawk tests/tools | Optional | S | | +| `xai-tty-utils` | 1.2k | TTY-safe spawn | hawk | Port | S | | +| `xai-hooks-plugins-types` | 1.2k | Hooks/plugins ACP DTOs | contracts | Port | S | | +| `xai-sqlite-journal` | 0.8k | SQLite journal mode | hawk/yaad/tok | Partial | XS | | +| `xai-system-power` | 0.7k | Sleep/wake notify | hawk sleep_prevent | Partial | S | | +| `xai-grok-http` | 0.6k | Shared HTTP client | hawk/netutil | Partial | XS | | +| `xai-agent-lifecycle` | 0.6k | Lifecycle hooks data | hawk hooks/engine | Partial | S | | +| `xai-gix-status` | 0.6k | Fast git status | hawk git tools | Partial | S | | +| `xai-grok-paths` | 0.6k | AbsPath types | hawk | N/A | XS | Go path.Clean enough | +| `xai-grok-secrets` | 0.6k | Secrets helpers | tok + eyrie | Partial | S | | +| `xai-grok-announcements` | 0.4k | Release announcements | hawk tips/notify | Port | S | | +| `xai-grok-auth` | 0.4k | Auth seam | eyrie + hawk auth | Partial | M | Browser OAuth optional | +| `xai-token-estimation` | 0.3k | Bytes/4 heuristic | **tok** | Skip | — | tok superior | +| `xai-tracing-macros` | 0.2k | Log macros | hawk observability | N/A | XS | | +| `xai-grok-env` | 0.2k | Backend env presets | eyrie/hawk | Partial | XS | | +| `xai-prompt-queue` | 0.2k | Prompt queue types | hawk | Port | S | | +| `xai-mixpanel` | 0.1k | Mixpanel client | — | **Skip** | — | Privacy; OTEL opt-in | +| `xai-grok-version` | 0.1k | Version | hawk VERSION | Done | — | | +| `xai-grok-models` | 0.1k | Default model IDs | eyrie catalog | Done* | — | | + +### 2.2 Common crates (tool protocol / hub / compaction) + +| Grok crate | ~LOC | Capability | Target | Status | Effort | Notes | +|------------|-----:|------------|--------|--------|--------|-------| +| `xai-computer-hub-sdk` | 14k | Remote tool server SDK | new or hawk | Port (Y2+) | XL | Only if multi-host tools wanted | +| `xai-computer-hub-core` | 4.2k | Transport/registry | same | Port (Y2+) | L | | +| `xai-computer-hub-mcp-adapter` | 1k | MCP into hub | hawk-mcpkit | Port (Y2+) | M | | +| `xai-tool-protocol` | 6.6k | Wire protocol | contracts + hawk | Port (Y2) | L | | +| `xai-tool-runtime` | 5.4k | Tool trait runtime | hawk tool | Partial | L | | +| `xai-tool-types` | 3.6k | Spawn/task types | **contracts** | Port | M | **P0** | +| `xai-grok-compaction` | 6.8k | Compaction engine | hawk engine + tok | Partial | L | | +| `xai-circuit-breaker` | 2.2k | HTTP breaker | eyrie/resilience | Partial | S | | +| `xai-tracing` | 0.8k | Tracing | hawk OTEL | Partial | S | | +| `xai-test-utils` | 0.4k | Hermetic git tests | hawk testutil | Partial | S | | +| `xai-interjection-core` | 0.3k | Interjection messaging | hawk | Port | S | Mid-turn user inject | +| `xai-proto-build` | — | Build tooling | — | Skip | — | Rust build | + +\*Done* = domain covered by hawk engine with different API. + +--- + +## 3. Tool-by-tool port matrix (Grok tools) + +### 3.1 Grok Build native tools + +| Grok tool | Hawk equivalent | Status | Port work | +|-----------|-----------------|--------|-----------| +| `bash` / `run_terminal_command` | `Bash` | Partial | background flag, timeout, safe-bash, kill | +| `read_file` | `Read` | Partial | media/PDF page ranges parity | +| `search_replace` | `Edit` | Partial | hashline / strict range policy optional | +| `list_dir` | `LS` | Done | | +| `grep` | `Grep` | Done | | +| `web_search` | `WebSearch` | Done | | +| `web_fetch` | `WebFetch` / AgenticFetch | Partial | SSRF hardening audit | +| `todo` | `TodoWrite` | Done | | +| `task` (spawn) | `Agent` | **Port** | full schema §A | +| `task_output` / wait | bg managers | **Port** | unify + tools | +| `kill_task` | partial | **Port** | | +| `monitor` | missing | **Port** | line stream + rate limit | +| `scheduler` | `Cron*` | Partial | `/loop` UX | +| `ask_user_question` | `AskUserQuestion` | Partial | multi-option UI | +| `enter_plan_mode` | `/spec` + plan | Partial | unify | +| `exit_plan_mode` | ApproveImplementation | Partial | | +| `update_goal` | todos/mission | Partial | goal tool | +| `lsp` | `LSP` / MCPLSP | Partial | | +| `image_gen` | missing | Port / Optional | provider-backed | +| `image_edit` | missing | Port / Optional | | +| `video_gen` | missing | Port / Optional | xAI-specific → multi-provider later | +| skills invoke | `Skill` | Partial | multi-harness discovery | +| MCP meta tools | MCP tools | Partial | search_tool / use_tool patterns | + +### 3.2 Codex / OpenCode compatibility tool ports (in Grok tree) + +Grok vendors codex/opencode tool implementations for compatibility profiles. + +| Compat surface | Hawk | Action | +|----------------|------|--------| +| Codex apply_patch / read / list / grep | hawk tools | Optional compat profile | +| OpenCode read/write/edit/bash/glob/grep/todo/skill | hawk | Optional `compat.opencode` | +| Claude import | missing | **Port** into trace/hawk | +| Cursor skills scan | missing | **Port** | + +--- + +## 4. Slash commands & UX (pager) + +Grok slash modules (port checklist → hawk `/` commands or CLI): + +| Grok slash | Status in hawk | Action | +|------------|----------------|--------| +| help | Partial | Port completeness | +| model / effort | Partial | Port effort levels | +| theme / vim_mode | Partial | Port gaps | +| compact | Partial | | +| config_agents / personas | Partial | Port agents modal parity | +| plugins / mcps / marketplace | Partial | Port | +| plan / view_plan | Partial | Align with /spec | +| loop | Partial (cron) | Port `/loop` | +| remember / memory | Partial (yaad) | Port UX | +| voice | Partial | Streaming STT | +| dashboard | Partial (HUD) | Port | +| usage | Partial | Port | +| import_claude | Missing | Port | +| fork / rewind / resume / rename | Partial | Port gaps | +| queue | Missing | Port prompt queue | +| btw (interjection) | Missing | Port | +| always_approve / auto | Partial | Map autonomy | +| login/logout | N/A keychain | Optional OAuth | +| share | Partial | | +| export / transcript | Partial | | +| announcements / release_notes | Missing | Port | +| feedback | Partial | | +| privacy | Partial | Port | +| terminal_setup | Missing | Port | +| timestamps | Partial | | +| history / find / copy | Partial | | +| imagine / imagine_video | Missing | Optional media tools | +| session_info / recap | Partial | | +| home / cd / new / exit | Partial | | +| multiline / mouse / screen_mode | Partial | TUI polish | +| scroll_debug / gboom | Dev only | Optional | + +--- + +## 5. User-guide docs (port as hawk user-guide) + +| Grok doc | Port to | +|----------|---------| +| 01–05 essentials | `hawk/docs/user-guide/` | +| 06 theming | same | +| 07 MCP | same + mcp-servers.md merge | +| 08 skills | same | +| 09 plugins | same | +| 10 hooks | same | +| 11 custom models | eyrie + hawk | +| 12 project rules AGENTS.md | exists Partial | +| 13 memory | yaad UX | +| 14 headless | CLI flags doc | +| 15 agent/ACP | acp doc | +| 16 subagents | spawn doc | +| 17 sessions | session doc | +| 18 sandbox | sandbox.toml doc | +| 19 plan mode | /spec doc | +| 20 background tasks | taskruntime doc | +| 21 terminal support | same | +| 22 permissions | autonomy+pipeline doc | +| 23 dashboard | same | +| 24 monitoring usage | cost/usage + cloud | + +**Effort:** M continuous across phases (write as features land). + +--- + +## 6. Small systems often missed (explicit) + +These are “small” crates/modules but required for **full** port claims: + +| Item | Grok home | Hawk action | Effort | +|------|-----------|-------------|--------| +| Folder trust | `workspace/folder_trust.rs` | **Port** `internal/trust` | M | +| envrc / direnv load | `workspace/envrc.rs` | **Port** | S | +| Foreign sessions (Claude/Codex) | `foreign_sessions/` | **Port** via trace | L | +| Managed config sync | `shell/managed_config` | **Port** via hawk-cloud | L | +| Claude import | `shell/claude_import*` | **Port** | M | +| Prompt queue | `xai-prompt-queue` | **Port** | S | +| Interjection (“btw”) | `xai-interjection-core` | **Port** | S | +| Announcements | `xai-grok-announcements` | **Port** | S | +| Crash handler | `xai-crash-handler` | **Port** | S | +| Auto-update | `xai-grok-update` | Complete Partial | M | +| Mermaid PNG | `xai-grok-mermaid` | **Port** (Go/SVG or subprocess) | M | +| Voice streaming | `xai-grok-voice` | Upgrade Partial | M | +| PTY control | `ptyctl` | **Port** for harness/tools | M | +| TTY-safe child spawn | `xai-tty-utils` | **Port** | S | +| Sleep/wake on auth | `xai-system-power` | Complete Partial | S | +| Hashline edit protocol | tools hashline | Optional Port | M | +| Leader/cluster multi-agent UI | pager leader_cluster | Optional (mission covers) | L | +| Relay / remote agent WS | shell relay/remote | Port for ACP serve | L | +| Upload / share pipeline | shell upload | Partial + privacy | M | +| Heap profile | shell heap_profile | Optional dev | S | +| Diag server | workspace diag_server | Optional | S | +| Preview supervisor | workspace | Optional | M | +| Hub auth/channel/server | workspace hub* | Y2 computer-hub | XL | +| Campaigns / version_overrides config | config | Port if enterprise | M | +| MDM macOS managed prefs | config macos_managed | Optional enterprise | M | +| Signed requirements fail-closed | config signed_policy | Port with cloud | L | +| Skills ignore/disabled/paths | skills config | Port | S | +| Compat cursor/claude toggles | config compat | Port | S | +| Safe-bash allowlist | permissions docs | Port | M | +| dontAsk / acceptEdits modes | permissions | Port | M | +| Dashboard view | pager views/dashboard | Port polish | M | +| Notifications system | pager notifications | Partial | S | +| Tips | pager tips | Partial | XS | +| Client identity | pager | Partial | XS | +| Inline media ffmpeg | pager | Optional | M | +| Hyperlink routing | pager | Port | S | +| Config TOML live edit | pager | Port | S | +| Goal classifier | shell session | Port | S | +| Trace classifier | shell | map to hawk trace | S | +| Repo changes tracking | shell session | Partial | M | +| Active sessions multi | shell | Partial | M | +| MCP doctor | shell mcp_doctor | Port | S | +| Bundle / extensions suggest | shell extensions | Partial | M | +| Instrumentation / inspect modules | shell | map observability | S | + +--- + +## 7. Year-by-year program (full port) + +### Year 0 — Foundation (months 0–12) + +**Theme:** Model can do what Grok’s agent control plane does; secure by default. + +| Quarter | Deliverables | Repos | +|---------|--------------|-------| +| **Q1** | Contracts spawn DTOs; wire explore/plan/general; stop hardcoded explore; explore bash hard gate; unify taskruntime; structured SpawnResult | contracts, hawk | +| **Q2** | sandbox.toml; folder trust; safe-bash; permission pipeline hooks-first; PreToolUse deny | hawk | +| **Q3** | File+HTTP hooks; vendor aliases; multi-harness skills; multi-component plugins; marketplace MVP | hawk, community-skills | +| **Q4** | Monitor/Wait/Kill; /loop; structured AskUser; plan/spec alignment; user-guide 01–12; crash handler; announcements; prompt queue; interjection | hawk | + +**Exit Year 0:** Contributor-ready “Grok-class agent controls” without claiming full TUI parity. + +### Year 1 — Integration & product depth (months 12–24) + +| Quarter | Deliverables | +|---------|--------------| +| **Q5** | ACP session/load/resume; richer updates; OpenAPI; sdk-go/python fields | +| **Q6** | Managed policy (signed) hawk-cloud → hawk; IT tier; config layer order | +| **Q7** | Foreign session import (Claude/Codex); envrc; hunk tracker; fast worktree; mermaid render | +| **Q8** | Voice streaming upgrade; update checker parity; MCP doctor; slash parity batch 1; user-guide 13–24 | + +### Year 2 — Workspace / hub / polish (months 24–36) + +| Focus | Items | +|-------|--------| +| Computer hub (optional product decision) | tool-protocol, hub server/client, remote tools, MCP adapter | +| TUI parity program | dashboard, modals, mouse, minimal mode, notifications polish | +| Compat | Claude/Cursor/OpenCode/Codex profiles | +| Media tools | image_gen/edit/video if multi-provider available | +| Relay/WS agent serve | remote IDE/web | +| Performance | CoW worktrees, git status, compaction quality | + +### Year 3–5 — Continuous parity + +- Diff audit bot: scan Grok public releases → open hawk issues +- Enterprise: MDM, SCIM (cloud), fleet policies +- Full editor suite (VS Code/Zed via ACP) +- Hardening, fuzz, SLSA releases +- Deprecate temporary adapters + +--- + +## 8. Phase implementation packs (engineering) + +Each pack is a shippable program of work with Definition of Done. + +### PACK-00: Inventory & gates (1–2 weeks) + +- [ ] Freeze this matrix in repo +- [ ] ADR: “Full behavioral port, Go multi-repo” +- [ ] Feature-flag registry +- [ ] CI job: `docs/plans` status table lint (optional later) + +### PACK-01: Contracts (2–3 weeks) — **start here** + +**Repo:** `hawk-core-contracts` + +- [ ] `agent` package: CapabilityMode, IsolationMode, SubagentType, SpawnRequest, SpawnResult +- [ ] Hook event name constants +- [ ] Tests for aliases and validation +- [ ] Version release + +**DoD:** hawk can import types; no engine deps. + +### PACK-02: Spawn control plane (6–8 weeks) + +**Repo:** `hawk` + +- [ ] Change `AgentSpawnFn` to SpawnRequest/Result +- [ ] Agent tool full schema (type, capability, isolation, resume, cwd, model, thoroughness, description, background) +- [ ] Wire plan/general (not only explore) +- [ ] Worktree isolation lifecycle +- [ ] Transcript resume_from +- [ ] MultiAgent typed tasks +- [ ] Explore bash AST allowlist +- [ ] Unify BackgroundAgentManager / BackgroundRunner / BackgroundAgentPool → `taskruntime` +- [ ] E2E + race tests + +**DoD:** Model can spawn plan agent that cannot Write; resume works. + +### PACK-03: Trust & sandbox (5–7 weeks) + +- [ ] `internal/trust` folder trust store + CLI +- [ ] Gate project hooks/MCP/LSP/plugins +- [ ] sandbox.toml loader (additive project merge) +- [ ] Profiles: off, workspace, read-only, strict, devbox, custom +- [ ] Deny globs fail-closed +- [ ] OS matrix tests + +### PACK-04: Hooks complete (4 weeks) + +- [ ] Event set expansion +- [ ] Claude/Cursor aliases +- [ ] File discovery paths +- [ ] HTTP runner +- [ ] Inject into PermissionEngine before autonomy +- [ ] Plugin env HAWK_PLUGIN_ROOT/DATA + +### PACK-05: Extensions (8–10 weeks) + +- [ ] Multi-component plugin layout +- [ ] Discovery scopes +- [ ] Marketplace multi-source install +- [ ] community-skills index schema for plugins +- [ ] Multi-harness skill scan +- [ ] Audit hardening + +### PACK-06: Task tools & loop (4 weeks) + +- [ ] GetTaskOutput, WaitTasks, KillTask tools +- [ ] Monitor tool (line processor, rate limit, auto-kill) +- [ ] `/loop` over cron with caps/expiry +- [ ] Shell background demote hotkey (optional TUI) + +### PACK-07: Plan & AskUser (3 weeks) + +- [ ] Structured AskUserQuestion UI +- [ ] enter/exit plan alignment with /spec +- [ ] update_goal tool if needed + +### PACK-08: Small UX systems batch (4 weeks) + +- [ ] Crash handler +- [ ] Announcements +- [ ] Prompt queue +- [ ] Interjection (btw) +- [ ] MCP doctor +- [ ] Terminal setup slash +- [ ] Privacy / usage docs + +### PACK-09: ACP + SDKs (7 weeks) + +- [ ] session/load, richer updates +- [ ] OpenAPI fields +- [ ] sdk-go + sdk-python +- [ ] Optional WS serve/relay + +### PACK-10: Enterprise policy (6 weeks) + +- [ ] Signed managed policy schema (cloud) +- [ ] hawk apply layers +- [ ] fail-open vs fail-closed +- [ ] IT non-excludable tier end-to-end + +### PACK-11: Session ecosystem (6 weeks) + +- [ ] Foreign sessions import (trace) +- [ ] Claude import CLI +- [ ] envrc load +- [ ] Hunk attribution tracker +- [ ] Fast worktree (CoW where available) + +### PACK-12: Media & mermaid (4–6 weeks) + +- [ ] Mermaid render path +- [ ] Image gen/edit if providers allow +- [ ] Video optional +- [ ] Inline media handling + +### PACK-13: Voice & power (3 weeks) + +- [ ] Streaming STT pipeline +- [ ] Sleep/wake aware auth refresh + +### PACK-14: TUI parity program (ongoing XL) + +Track Grok pager features against hawk Bubble Tea: + +- [ ] Dashboard view parity +- [ ] Agents/personas modal +- [ ] Plugins/marketplace modal +- [ ] Minimal mode +- [ ] Mouse/scroll polish +- [ ] Notifications +- [ ] Theme completeness +- [ ] Slash completeness audit (table §4) + +### PACK-15: Computer hub (optional Y2 XL) + +Only if product requires remote tool hosts: + +- [ ] tool-protocol Go +- [ ] hub server/client +- [ ] MCP adapter +- [ ] workspace RPC + +### PACK-16: Docs full port (parallel) + +- [ ] `docs/user-guide/01` … `24` +- [ ] Architecture notes stay separate + +### PACK-17: Continuous parity (forever) + +- [ ] Quarterly grok-build public tree diff +- [ ] Issue auto-filing for new Grok features +- [ ] Security review cadence + +--- + +## 9. Per hawk-eco repo full ownership checklist + +### `hawk` (majority) + +Port/finish: spawn, tools, sandbox, trust, hooks, plugins, marketplace client, TUI, ACP, headless, slash, taskruntime, hunks, mermaid, voice, crash, announcements, queue, interjection, plan, ask user, update, PTY harness, envrc, permissions pipeline, user-guide. + +### `hawk-core-contracts` + +Port: spawn/capability/isolation types, hook events, optional sandbox policy DTO, tool protocol types (later). + +### `eyrie` + +Absorb: sampling/stream/retry/circuit-breaker lessons; catalog already ahead; managed deployment hooks; **do not** become Grok-only auth. + +### `yaad` + +Port memory **UX**: dream, toggle priority, flush; keep graph store. + +### `tok` + +Keep; optionally align display token estimates; secrets scanning already stronger. + +### `trace` + +Foreign session import; share/export; session indexing parity with Grok session features that are capture-oriented. + +### `sight` / `inspect` + +No direct Grok crates; keep peer engines; ensure hawk composition matches Grok “review/inspect” product moments if any. + +### `hawk-mcpkit` + +Liveness, shared OAuth helpers, hub MCP adapter if hub lands. + +### `hawk-community-skills` + +Marketplace content + multi-component packages + validation. + +### `hawk-cloud` + +Managed policy, usage ledger (exists), entitlements, SSO/SCIM (exists primitives). + +### `hawk-sdk-go` / `hawk-sdk-python` + +Spawn options, plugins meta, permission/sandbox, stream formats. + +### `graycode-core` + +Dashboard/usage/marketplace web UI only. + +--- + +## 10. Explicit Skip list (still “accounted for”) + +| Grok thing | Why skip / transform | +|------------|----------------------| +| Entire Rust monorepo layout | Wrong language/org model | +| Mixpanel default | Privacy | +| Sentry hard-dependency | Opt-in error reporting | +| SpaceXAI-only browser auth as sole path | Multi-provider + keychain | +| `xai-token-estimation` | tok wins | +| Replace yaad with markdown memory files | Graph better | +| Replace eyrie with xai-grok-sampler | Multi-provider better | +| ratatui widgets | Bubble Tea stack | +| `xai-proto-build` | Rust build | +| Closed contribution policy | Hawk is open | + +Skipping is **documented completion**, not unfinished work. + +--- + +## 11. Tracking spreadsheet schema (use in GitHub Projects) + +Columns: + +1. ID (e.g. PACK-02-07) +2. Grok source (crate/path) +3. Capability name +4. Target repo +5. Target package +6. Status (Done/Partial/Port/Skip) +7. Pack +8. Effort +9. Priority (P0–P3) +10. Owner +11. PR links +12. Tests (unit/integ/OS) +13. User-guide section +14. Feature flag +15. Notes + +Import every row from §2, §3, §4, §6. + +--- + +## 12. Definition of “full port complete” + +Declare complete when: + +1. Every row in §2 is Done, Partial-accepted (with written gap < 10% behavior), or Skip. +2. Every tool in §3 is Done or Explicitly Optional. +3. Every slash in §4 is Done or Explicitly Dev-only. +4. User-guide 01–24 exists and matches behavior. +5. ACP multi-turn + permissions works in at least one external editor. +6. Folder trust + sandbox deny globs pass security tests on macOS+Linux. +7. Marketplace can install a multi-component plugin. +8. Subagent explore/plan/general + resume + worktree isolation work. +9. No second memory/token/provider stack duplicated against yaad/tok/eyrie. +10. Continuous parity process (PACK-17) running for 2 releases. + +--- + +## 13. Resource model for “all including small things” + +| Team size | Time to Year-0 exit | Time to full port claim | +|-----------|---------------------|-------------------------| +| 1 senior | ~12 mo | ~4–5 years | +| 2 seniors | ~7–9 mo | ~2.5–3.5 years | +| 3–4 (core+cloud+skills) | ~6 mo | ~2–3 years | + +**Bottleneck:** TUI parity (pager 415k LOC class) and optional computer-hub — not the spawn contracts. + +--- + +## 14. Immediate next 90 days (actionable) + +| Week | Work | +|------|------| +| 1–2 | PACK-01 contracts + ADR | +| 3–8 | PACK-02 spawn control plane | +| 9–12 | Start PACK-03 trust/sandbox (parallel safe-bash) | + +Do **not** start marketplace or computer-hub before PACK-02/03. + +--- + +## 15. Relationship to shorter plan + +| Document | Scope | +|----------|--------| +| `GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md` | Agent control + trust + hooks + marketplace focus (~12–18 mo) | +| **This document** | **Every crate, tool, slash, small system** → multi-year full port | + +Execute shorter plan as **Year 0** of this master plan. + +--- + +## 16. One-line summary + +**Port all of grok-eco into hawk-eco = reimplement Grok Build’s full capability surface in Go across hawk-eco repos, map each crate to an owner engine, skip vendor/privacy conflicts, wire Hawk’s existing partial systems first, and run a multi-year program ending in behavioral parity—not a Rust code transplant.** + +--- + +*Maintainers: update Status column as PRs merge; re-run crate inventory when grok-build updates.* diff --git a/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md b/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md new file mode 100644 index 00000000..af50db42 --- /dev/null +++ b/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md @@ -0,0 +1,994 @@ +# Hawk-Eco Long-Horizon Plan: Grok-Class Agent Control Plane (Go) + +**Status:** Planning +**Date:** 2026-07-16 +**Horizon:** ~12–18 months (phased; can compress with parallel teams) +**Language:** Go only (no Rust ports; reimplement contracts and behavior) +**Primary product:** `hawk` +**Supporting repos:** `hawk-core-contracts`, `eyrie`, `yaad`, `tok`, `trace`, `sight`, `inspect`, `hawk-mcpkit`, `hawk-community-skills`, `hawk-cloud`, `hawk-sdk-go`, `hawk-sdk-python`, `graycode-core` (UI only) + +--- + +## 0. Executive summary + +### Goal + +Bring Hawk to **Grok-class agent control-plane quality** (typed subagents, sandbox profiles, folder trust, hooks, plugins/marketplace, ACP depth, monitor/scheduler, enterprise managed policy) **without** abandoning Hawk’s multi-repo Go platform advantages (eyrie multi-provider, yaad graph memory, tok compression, mission mode, contracts, cloud ledger). + +### Non-goals + +- Rewrite Hawk in Rust or monorepo-collapse engines into hawk. +- Vendor lock-in to a single LLM host auth model. +- Replace eyrie/yaad/tok/trace with Grok-shaped internal crates. +- Default opt-in product telemetry that violates privacy-first posture. + +### Strategy + +1. **Close wiring gaps first** — Hawk already has many *types* and *partial systems* that are not exposed on the Agent tool or unified. +2. **Stabilize contracts** in `hawk-core-contracts` before multi-repo consumers. +3. **Ship vertical slices** (end-to-end user value) every quarter. +4. **Keep engines peer-independent**; hawk remains the only product orchestrator. + +--- + +## 1. Deep audit: current state (verified 2026-07-16) + +### 1.1 What already exists (do not rebuild) + +| Area | Evidence | Maturity | +|------|----------|----------| +| Subagent **modes as types** | `internal/engine/agent/agent_types.go`: `explore`, `general`, `plan`; thoroughness quick/medium/very-thorough; `ExploreTools` / `PlanTools` | **Types strong** | +| Mode tool allowlists + budgets | `internal/engine/agent/subagent_budget.go`: `ModeToolAllowlist`, `FilterToolsForMode`, turn budgets | **Library strong** | +| Subagent spawn implementation | `internal/engine/agent_session_tool.go`: `spawnSubAgent`, model cascade, tool filter, max depth, inherits spec stage | **Partial** | +| Agent tool surface | `internal/tool/agent.go`: only `prompt`, `run_in_background`, `agent_id`, `retry_of` | **Weak vs types** | +| Default wire path | `WireAgentTool()` **always** calls `spawnSubAgent(..., SubAgentExplore, 0)` | **Critical gap** | +| Background systems (3+) | `tool.BackgroundAgentManager`, `engine.BackgroundRunner`, `agent.BackgroundAgentPool` | **Duplicated** | +| Permission + spec gate | `engine/safety/permission_engine.go`: DryRun → SpecStage → autonomy → classifier → auto-mode → memory → prompt | **Strong core** | +| Sandbox backends | seatbelt, landlock, seccomp, docker, gvisor; modes strict/workspace/off | **Strong backends, weak profiles** | +| Personas | `multiagent/agents`: ReadOnly, tools, hooks, YAML | **Strong** | +| Mission worktrees | `multiagent/worker.go` creates git worktrees | **Strong for missions only** | +| Worktree tools | `EnterWorktree` / `ExitWorktree` | **User-driven, not spawn isolation** | +| Plugins V1/V2 | subprocess/daemon/WASM, hooks on manifest | **Partial packaging** | +| Skills install + community registry | git install + huge `hawk-community-skills` | **Content strong** | +| ACP first cut | `internal/acp` + `cmd/acp.go`: init/new/prompt/cancel/permission | **Thin** | +| Session checkpoints/fork | `internal/session/checkpoint.go`, fork, export | **Strong** | +| Cron | `internal/tool/cron.go` max 256 jobs | **Present, UX thin** | +| Cloud enterprise policy DTO | `hawk-cloud` enterprise: model allow/deny, capability allow/deny | **Primitive exists** | +| Contracts | severity, tools, events, policy, review, verify, sessions phases | **Missing agent spawn types** | +| Compaction | multiple `engine/compact*.go` | **Present** | +| Memory | yaad graph engine | **Architecturally ahead of Grok markdown memory** | +| Token/cost | tok library | **Architecturally ahead of Grok heuristic** | + +### 1.2 Critical gap pattern + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Types & budgets exist (explore/plan/general) │ +│ spawnSubAgent(mode) exists │ +│ WireAgentTool hardcodes explore │ +│ Agent tool schema cannot pass mode / isolation / resume │ +│ → Model cannot use plan mode or general-purpose by design │ +└─────────────────────────────────────────────────────────────┘ +``` + +This is the single highest-ROI fix: **wire what already exists**, then extend. + +### 1.3 Permission pipeline (actual vs target) + +**Actual order** (`PermissionEngine.CheckTool`): + +1. DryRun deny-all +2. Spec stage gate (independent of autonomy) +3. Autonomy preset / “safe tool” short-circuit +4. Bypass killswitch +5. Bash classifier “safe” +6. AutoMode +7. Permission memory +8. User prompt + +**Missing relative to Grok-class:** + +- PreToolUse **hooks before** all of the above (hooks exist as registry but not as first deny gate in this path) +- Explicit ordered **deny rules** package as distinct stage +- Segment-aware **safe-bash allowlist** (classifier is different, weaker documentation) +- Named modes `dontAsk` / `acceptEdits` as first-class (partially in jsonc `defaultMode`) +- Folder trust gate for project automation + +### 1.4 Duplication / debt that the plan must fix early + +| Debt | Risk if ignored | +|------|-----------------| +| 3 background task managers | Inconsistent ids, race bugs, broken Wait/Kill | +| Agent tool signature `func(ctx, prompt string)` | Blocks mode/isolation/model forever | +| explore Bash allowlisted without hard read-only bash AST gate | Explore mode can still mutate via shell | +| Plugin = tools binary, not multi-component package | Marketplace cannot ship team bundles | +| No folder trust | Project hooks/MCP = RCE vector | + +--- + +## 2. Target architecture (Go) + +```text + ┌──────────────────────────────┐ + │ hawk TUI / CLI / daemon/ACP │ + └──────────────┬───────────────┘ + │ + ┌─────────────────────────┼─────────────────────────┐ + ▼ ▼ ▼ +┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ SpawnController │ │ Trust & Policy │ │ Extension Host │ +│ typed subagents │ │ folder trust │ │ plugins/skills │ +│ isolation │ │ sandbox.toml │ │ marketplace │ +│ resume │ │ hooks pipeline │ │ MCP/LSP load │ +└────────┬────────┘ └────────┬─────────┘ └────────┬─────────┘ + │ │ │ + ▼ ▼ ▼ +┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ engine.Session │ │ PermissionEngine │ │ plugin.Manager │ +│ + TaskRuntime │ │ + HookRunner │ │ skill.Loader │ +└────────┬────────┘ └──────────────────┘ └──────────────────┘ + │ + ┌────┴────┬──────────┬──────────┬──────────┐ + ▼ ▼ ▼ ▼ ▼ + eyrie yaad tok trace sight/inspect +``` + +**New core package (proposed):** `hawk/internal/spawn` (or expand `engine/agent`) owning: + +- `SpawnRequest` / `SpawnResult` (Go structs aligned with contracts) +- capability filter, isolation worktree lifecycle +- transcript persistence for resume +- unified task registry (shell bg + subagents + monitors) + +**Contracts package (proposed):** `hawk-core-contracts/agent` (or `spawn`): + +- enums + DTOs only; no engine imports + +--- + +## 3. Concept inventory → implementation status + +Legend: **Done** | **Partial** | **Missing** | **N/A (keep engine)** + +### Wave A — Agent control (P0) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| A1 | Typed spawn schema on Agent tool | Missing on tool; Partial in engine | Full | +| A2 | `subagent_type` explore/plan/general-purpose | Types Partial; wire Missing | Full | +| A3 | Thoroughness for explore | Types Done; wire Missing | Full | +| A4 | Capability modes read-only/read-write/execute/all | Partial (modes only) | Full enum + filter | +| A5 | Isolation none/worktree | Mission Partial; Agent Missing | Full | +| A6 | True `resume_from` transcript | Missing (agent_id status only) | Full | +| A7 | Structured SpawnResult | Partial envelope | Full | +| A8 | MultiAgent typed task objects | Missing | Full | +| A9 | Unify background task runtime | Partial (3 systems) | Full | +| A10 | Explore Bash hard read-only | Partial (comment only) | Full (AST allowlist) | + +### Wave B — Trust & sandbox (P1) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| B1 | Built-in profiles incl. read-only, devbox | Partial (3 modes) | Full | +| B2 | `sandbox.toml` extends/deny globs | Missing | Full | +| B3 | Project additive-only profile merge | Missing | Full | +| B4 | Folder trust store | Missing | Full | +| B5 | Gate hooks/MCP/LSP/plugins on trust | Missing | Full | +| B6 | Safe-bash allowlist (segment-aware) | Partial classifier | Full | +| B7 | Permission pipeline doc + hooks first | Partial | Full | +| B8 | Named modes acceptEdits/dontAsk | Partial jsonc | Full product | + +### Wave C — Hooks (P2) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| C1 | Full event set (subagent, stop, failure) | Partial | Full | +| C2 | Vendor event aliases | Missing | Full | +| C3 | File-discovered hooks | Partial (in-process) | Full | +| C4 | HTTP hooks | Missing | Full | +| C5 | Plugin env HAWK_PLUGIN_ROOT/DATA | Missing | Full | +| C6 | PreToolUse can deny in CheckTool path | Missing | Full | + +### Wave D — Plugins & marketplace (P3) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| D1 | Multi-component plugin layout | Missing | Full | +| D2 | Discovery scopes + priority | Partial | Full | +| D3 | Marketplace multi-source | Design only | Full | +| D4 | Multi-harness skill scan | Missing | Full | +| D5 | Audit on install (keep) | Partial | Harden | + +### Wave E — Tasks / monitor / scheduler (P4) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| E1 | Shell background task_id | Partial | Full | +| E2 | GetTaskOutput / WaitTasks / KillTask | Missing as tools | Full | +| E3 | Monitor line-stream tool | Missing | Full | +| E4 | /loop UX over cron | Partial cron | Full | + +### Wave F — Plan UX & AskUser (P5) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| F1 | Align /spec with plan subagent | Partial | Full | +| F2 | Structured AskUserQuestion | Partial free-text | Full | + +### Wave G — ACP & SDKs (P6) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| G1 | session/load + resume | Missing | Full | +| G2 | Richer session/update | Partial | Full | +| G3 | Optional agent serve/WS | Missing | Optional | +| G4 | SDK spawn/plugin fields | Missing | Full | +| G5 | OpenAPI sync | Partial | Full | + +### Wave H — Sessions / import / attribution (P7) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| H1 | Foreign session import | Missing | Full (trace) | +| H2 | Hunk agent vs external | Missing | Full | +| H3 | Checkpoint already | Done | Keep | + +### Wave I — Memory UX (P8) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| I1 | Toggle priority order | Partial | Full | +| I2 | Dream/consolidate product UX | Partial yaad | Full | +| I3 | Graph memory core | Done (yaad) | Keep | + +### Wave J — Enterprise (P9) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| J1 | Config layer merge order | Partial | Full | +| J2 | Managed signed policy pull | Partial cloud DTO | Full | +| J3 | Fail-open default / fail-closed org | Missing | Full | + +### Wave K — Docs & polish (P10) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| K1 | User-guide tiered docs | Missing | Full | +| K2 | Mermaid render optional | Partial text | Optional | +| K3 | Keep engines independent | Done | Keep | + +--- + +## 4. Multi-quarter roadmap (long horizon) + +Assumes ~1–2 senior Go engineers on hawk core + fractional engine/cloud; scale parallelizes quarters. + +```text +Q1 Foundation: contracts + spawn wiring + background unify +Q2 Trust: sandbox.toml + folder trust + safe-bash + hooks gate +Q3 Extensions: plugins multi-component + marketplace MVP + skill multi-harness +Q4 Runtime: monitor/wait/loop + AskUser structured + plan alignment +Q5 Integration: ACP phase-2 + OpenAPI + SDKs +Q6 Enterprise: managed policy + cloud apply + IT tier +Q7 Ecosystem: foreign import (trace) + hunk attribution + memory UX +Q8 Hardening: perf, fuzz, security audit, docs completion, GA polish +``` + +Calendar can slip; **order of waves should not reverse** (contracts before marketplace consumers, trust before project plugins). + +--- + +## 5. Detailed phase plans + +### Phase 0 — Prep (1–2 weeks) + +**Objectives** + +- Freeze concept list (this document). +- Inventory all `AgentSpawnFn` call sites and background managers. +- Add ADR: “Typed spawn is the only subagent entrypoint.” + +**Deliverables** + +- [ ] ADR in `docs/architecture/adr/` +- [ ] Call-site map in this plan appendix +- [ ] Test plan template for spawn matrix +- [ ] Dependency freeze: no new third background system + +**Exit criteria** + +- Team agrees Wave A is first PR stack; no marketplace before folder trust. + +--- + +### Phase 1 — Contracts foundation (2–3 weeks) + +**Repo:** `hawk-core-contracts` +**Module:** new package `agent` (or `spawn`) — stdlib only + +#### 1.1 Types to add + +```go +// CapabilityMode +const ( + CapReadOnly = "read-only" + CapReadWrite = "read-write" + CapExecute = "execute" + CapAll = "all" +) + +// IsolationMode +const ( + IsoNone = "none" + IsoWorktree = "worktree" +) + +// SubagentType +const ( + TypeGeneralPurpose = "general-purpose" // maps to general + TypeExplore = "explore" + TypePlan = "plan" +) + +type SpawnRequest struct { + Prompt string + Description string + SubagentType string + CapabilityMode string // optional; derived from type if empty + Isolation string + ResumeFrom string + CWD string + Model string + Background bool + Thoroughness string // explore only + ParentSession string +} + +type SpawnResult struct { + SubagentID string + SubagentType string + Status string // running|completed|failed + Output string + Summary string + ToolCalls int + Turns int + DurationMs int64 + WorktreePath string + Persona string + Error string +} +``` + +#### 1.2 Work items + +| ID | Task | Effort | Tests | +|----|------|--------|-------| +| C-1 | Package + Parse helpers (aliases: general↔general-purpose, ReadOnly) | S | table tests | +| C-2 | Validate mutual exclusion cwd vs worktree | S | unit | +| C-3 | Version bump + CHANGELOG | S | CI module | +| C-4 | Proto optional later — **not required Q1** | — | — | + +**Exit criteria** + +- `go test ./...` green; hawk can depend on new pseudo-version without engine imports. + +--- + +### Phase 2 — Spawn controller & Agent tool (4–6 weeks) ★ highest ROI + +**Repo:** `hawk` +**Packages:** `internal/engine/agent`, `internal/engine`, `internal/tool` + +#### 2.1 Change `AgentSpawnFn` signature + +**Today:** + +```go +AgentSpawnFn func(ctx context.Context, prompt string) (string, error) +``` + +**Target:** + +```go +AgentSpawnFn func(ctx context.Context, req agentcontracts.SpawnRequest) (agentcontracts.SpawnResult, error) +``` + +Migration: temporary adapter for one release if needed; prefer single breaking change in 0.x. + +#### 2.2 Wire `WireAgentTool` correctly + +| Step | Change | +|------|--------| +| 1 | Parse Agent tool JSON → `SpawnRequest` | +| 2 | Map `subagent_type` → `SubAgentMode` (+ thoroughness) | +| 3 | Default capability from mode; allow override | +| 4 | Apply `FilterToolsForMode` **and** capability filter | +| 5 | Isolation worktree: create under `.hawk/worktrees/` using shared helper from mission worker | +| 6 | Persist child session transcript under `~/.hawk/subagents//` | +| 7 | `resume_from`: load transcript, append prompt, re-spawn with same type | +| 8 | Return structured `SpawnResult` JSON to model | + +#### 2.3 Explore Bash hard gate + +- Reuse `internal/tool` bash AST if present (`bash_ast.go`). +- For `IsReadOnlyMode`: only allowlist commands (Grok-class list). +- Deny `rm`, redirects, `git commit`, network tools, etc. + +#### 2.4 MultiAgent + +```json +{ + "tasks": [ + { + "prompt": "...", + "description": "scan auth", + "subagent_type": "explore", + "capability_mode": "read-only", + "thoroughness": "very-thorough" + } + ], + "run_in_background": true +} +``` + +#### 2.5 Unify background runtime + +**Target single package:** `internal/taskruntime` + +| Capability | API | +|------------|-----| +| Register | shell / subagent / monitor | +| Get | status + output | +| Wait | any/all + timeout | +| Kill | cancel context + process group | +| List | active | + +Deprecate: + +- Prefer migrating call sites off `BackgroundRunner` and `BackgroundAgentPool` into one manager; keep thin wrappers one release. + +#### 2.6 PR stack (Graphite-friendly) + +1. Contracts bump in hawk +2. `SpawnRequest` internal + adapter +3. Agent tool schema + parse +4. Wire modes plan/general/explore +5. Worktree isolation +6. Transcript resume +7. Background unify +8. Explore bash hard gate +9. MultiAgent objects +10. Docs + e2e tests + +**Exit criteria** + +- Model can spawn plan agent that cannot Write. +- Explore cannot `rm` via Bash. +- Resume continues conversation with prior tool results present. +- Single task id namespace for bg subagents. +- Race tests pass under `-race`. + +**Effort:** ~6–8 eng-weeks. + +--- + +### Phase 3 — Sandbox profiles + folder trust (4–5 weeks) + +**Repo:** `hawk` +**Packages:** `internal/sandbox`, `internal/trust` (new), `internal/mcp`, `internal/hooks`, `internal/plugin` + +#### 3.1 sandbox.toml + +```toml +# ~/.hawk/sandbox.toml +[profiles.ci] +extends = "strict" +restrict_network = true +deny = ["**/.env", "**/*.pem", "**/*credentials*"] +``` + +**Loader rules (security-critical):** + +1. Load user global. +2. Load project `.hawk/sandbox.toml` **additive only** (new names only). +3. Warn on conflicting redefinition; ignore project redefinition. +4. Deny globs applied on top of backend (seatbelt/landlock/bwrap-equivalent). +5. Fail closed when deny requested but OS cannot enforce. + +#### 3.2 Profile matrix + +| Name | FS read | FS write | Network | Maps from today | +|------|---------|----------|---------|-----------------| +| off | all | all | all | ModeOff / TierOff | +| workspace | all | cwd+state+tmp | allow | ModeWorkspace | +| read-only | all | state+tmp only | deny where possible | **new name** | +| strict | cwd+sys | cwd+state+tmp | deny | ModeStrict | +| devbox | all | broad except protected | allow | **new** | +| custom | extends | extends | flag | **new** | + +#### 3.3 Folder trust + +**Store:** `~/.hawk/trusted_folders.toml` + +```toml +[[folders]] +path = "/Users/me/proj" +trusted_at = "2026-07-16T00:00:00Z" +``` + +**Commands:** `/trust`, `/trust status`, `/trust revoke`, CLI `--trust` + +**Gates when untrusted:** + +- Project hooks +- Project MCP +- Project LSP +- Project plugins +- Optional: project skills that execute scripts + +**Exit criteria** + +- Malicious project cannot redefine global `strict` profile. +- Untrusted project hooks never run (tested). +- Custom deny `**/.env` blocks Read and Bash cat (integration tests per OS). + +**Effort:** ~5–7 eng-weeks (OS matrix heavy). + +--- + +### Phase 4 — Hooks pipeline completion (3–4 weeks) + +**Repo:** `hawk` +**Package:** `internal/hooks` + +#### 4.1 Event model expansion + +Add: `stop`, `stop_failure`, `post_tool_failure`, `permission_denied`, `user_prompt_submit`, `notification`, `subagent_start`, `subagent_stop` +Keep existing; alias map for Claude/Cursor names. + +#### 4.2 Runners + +| Type | Behavior | +|------|----------| +| command | shell with timeout, env HAWK_* | +| http | POST JSON, timeout, optional deny on non-2xx for pre_tool | + +#### 4.3 Integration into PermissionEngine + +```text +CheckTool: + 1. DryRun + 2. PreTool hooks (deny wins) + 3. Spec stage + 4. Explicit deny rules + 5. ... rest +``` + +#### 4.4 File discovery + +- `~/.hawk/hooks/*.json` +- `.hawk/hooks/*.json` (trust required) +- Compat: `.claude/settings.json` hooks, `.cursor/hooks.json` behind flags + +**Exit criteria** + +- Pre-tool hook can block `Bash(rm -rf)` even in Autonomous. +- Project hook skipped until trust. +- HTTP hook timeouts do not hang session (fail open or closed by config). + +**Effort:** ~4 eng-weeks. + +--- + +### Phase 5 — Plugins multi-component + marketplace MVP (6–8 weeks) + +**Repos:** `hawk`, `hawk-community-skills` + +#### 5.1 Plugin layout (convention) + +```text +my-plugin/ + plugin.json # optional + skills/**/SKILL.md + commands/*.md + agents/*.yaml|*.md + hooks/hooks.json + .mcp.json + .lsp.json +``` + +Loader merges components; tools optional. + +#### 5.2 Discovery priority + +1. Session meta / SDK +2. `--plugin-dir` +3. Project `.hawk/plugins` (trust) +4. User `~/.hawk/plugins` +5. Config extra paths + +#### 5.3 Marketplace + +| Component | Owner | +|-----------|--------| +| Source list config | hawk | +| Index schema | community-skills + hawk | +| Install resolve (git) | hawk | +| Audit | hawk plugin malware_check (extend) | +| CLI `hawk plugins` / TUI tab | hawk | +| Optional web gallery | graycode-core later | + +#### 5.4 Multi-harness skills + +Scan `.claude`, `.cursor`, `.agents` with toggles in settings; dedupe by name. + +**Exit criteria** + +- Install plugin with skill+hook+MCP in one command. +- Marketplace install from official + one third-party source. +- Audit fails install of skill with shell exfil pattern. + +**Effort:** ~8–10 eng-weeks (product + content). + +--- + +### Phase 6 — Task runtime: monitor, wait, loop (3–4 weeks) + +Depends on Phase 2 unified `taskruntime`. + +| Tool | Spec | +|------|------| +| `GetTaskOutput` | id, optional timeout_ms | +| `WaitTasks` | ids[], mode any\|all, timeout | +| `KillTask` | id | +| `Monitor` | command, persistent?, timeout; line events → stream | + +**Loop UX** + +- `/loop 5m ` → cron wrapper with caps (max jobs, 7-day expiry) +- Fire immediate + interval + +**Exit criteria** + +- Dev server monitor produces line events without flooding (rate limit). +- Wait any returns when first subagent completes. + +**Effort:** ~4 eng-weeks. + +--- + +### Phase 7 — Plan alignment + structured AskUser (2–3 weeks) + +| Work | Detail | +|------|--------| +| Plan subagent | Default tool for “design only”; writes plan file under `.hawk/plans/` or specs | +| Spec workflow | Document mapping: plan agent → `/spec` stages; avoid double systems | +| AskUserQuestion | questions[], options, multi_select, other, cancel message | +| TUI | Reuse autonomy/spec pickers | + +**Exit criteria** + +- Structured multi-question UI in chat. +- Plan agent never writes production code tools. + +**Effort:** ~3 eng-weeks. + +--- + +### Phase 8 — ACP phase-2 + SDKs + OpenAPI (5–7 weeks) + +**Repos:** `hawk`, `hawk-sdk-go`, `hawk-sdk-python` + +| Milestone | Scope | +|-----------|--------| +| ACP-1 | session/load, list sessions | +| ACP-2 | richer updates (tool start/end, thoughts if available) | +| ACP-3 | permission round-trip hardened | +| API | OpenAPI fields: spawn options, sandbox, autonomy, plugins meta | +| SDK | AgentConfig + ChatRequest fields; plugin dirs | +| Optional | `hawk agent serve --bind` WS | + +**Exit criteria** + +- Zed/VS Code can drive hawk ACP for multi-turn with permissions. +- SDK e2e against daemon contract snapshot tests. + +**Effort:** ~7 eng-weeks (+ extension work separate). + +--- + +### Phase 9 — Enterprise managed policy (4–6 weeks) + +**Repos:** `hawk-cloud`, `hawk`, optional `eyrie` + +| Piece | Detail | +|-------|--------| +| Policy document | models allow/deny, capabilities, sandbox deny, max budget, tool denylist | +| Signing | Ed25519 envelope; hawk verifies before apply | +| Apply path | `~/.hawk/managed_policy.json` layers under user config | +| Default | fail-open for individual; org can require fail-closed | +| IT tier | non-excludable rules (already sketched in product — finish) | +| Cloud UI | graycode-core admin later | + +Build on existing enterprise policyInput (model/capability lists). + +**Exit criteria** + +- Org policy denies model X on device within TTL. +- Tampered signature rejected. +- Offline individual still works without cloud. + +**Effort:** ~6 eng-weeks. + +--- + +### Phase 10 — Ecosystem polish (trace import, hunks, yaad UX) (4–6 weeks) + +| Item | Repo | Detail | +|------|------|--------| +| Foreign import | trace + hawk CLI | Claude/Codex session metadata → index | +| Hunk attribution | hawk | agent vs external edits via fsnotify | +| Memory UX | yaad + hawk | toggle priority, `/dream` consolidate | +| Mermaid optional | hawk | sandbox render path | + +**Exit criteria** + +- Import at least Claude sessions list. +- “Agent changed files this turn” accurate in TUI. + +**Effort:** ~5 eng-weeks. + +--- + +### Phase 11 — Hardening & GA (ongoing last quarter) + +- Fuzz bash allowlist, sandbox deny globs, spawn JSON +- Race + stress multiagent +- Security review: trust, hooks, marketplace install +- User-guide `docs/user-guide/01–22` +- Performance: spawn latency, worktree creation +- Compatibility matrix: macOS seatbelt, Linux landlock, Windows best-effort +- Release: feature flags default-on after bake + +--- + +## 6. Cross-repo ownership matrix + +| Concept family | Primary | Secondary | Contracts? | +|----------------|---------|-----------|------------| +| Spawn / capability / isolation | hawk | — | yes | +| Sandbox profiles | hawk | — | optional DTO | +| Folder trust | hawk | — | no | +| Hooks | hawk | — | event names yes | +| Plugins / marketplace | hawk | community-skills | manifest schema | +| Multi-harness skills | hawk | community-skills | no | +| Monitor/tasks | hawk | — | optional | +| ACP | hawk | sdk-go/python | OpenAPI | +| Managed policy | hawk-cloud | hawk, graycode-core | yes DTO | +| Foreign import | trace | hawk | optional | +| Memory dream UX | yaad | hawk | no | +| Token | tok | — | no change | +| Providers | eyrie | hawk | no change | +| Review engines | sight/inspect | hawk | findings already | + +--- + +## 7. Testing strategy (all phases) + +### 7.1 Unit + +- Parse/validate SpawnRequest (aliases, mutual exclusion) +- FilterToolsForMode × capability matrix +- sandbox.toml merge security +- hook alias map +- safe-bash segments + +### 7.2 Integration + +- Spawn explore/plan/general end-to-end with fake LLM +- Worktree isolation: parent tree unchanged +- Resume transcript +- Untrusted project hooks never execute +- Deny glob blocks file read + +### 7.3 Race / stress + +- MultiAgent max concurrency +- Background collect +- Monitor rate limit + +### 7.4 OS matrix + +| Feature | macOS | Linux | Windows | +|---------|-------|-------|---------| +| Seatbelt profiles | required | n/a | n/a | +| Landlock/seccomp | n/a | required | n/a | +| Worktree isolation | yes | yes | best-effort | +| Folder trust | yes | yes | yes | + +### 7.5 Compatibility / contract CI + +- `hawk-core-contracts` version pin +- OpenAPI snapshot for SDKs +- Engine import boundary scripts (existing ecosystem boundary checks) + +--- + +## 8. Feature flags & rollout + +| Flag | Default early | GA | +|------|---------------|-----| +| `spawn.v2` | on for contributors | on | +| `sandbox.profiles` | off → on | on | +| `trust.folder` | on (secure default) | on | +| `hooks.file_discovery` | off → on | on | +| `plugins.marketplace` | off | on | +| `monitor.tool` | off | on | +| `acp.v2` | off | on | +| `policy.managed` | off | org-only | + +Prefer **secure defaults** (folder trust on) even if noisy. + +--- + +## 9. Risk register + +| Risk | Impact | Mitigation | +|------|--------|------------| +| Signature change of AgentSpawnFn breaks plugins | High | Adapter release; version min_hawk | +| Worktree disk bloat | Med | GC old worktrees; symlink shared dirs (already pattern) | +| Explore still escapes via Bash | High | AST allowlist + sandbox | +| Marketplace supply chain | High | audit, pin commit SHAs, signatures later | +| Hook RCE via project | Critical | folder trust + fail closed | +| Three bg systems mid-migration | Med | single package + deprecation window | +| Over-scoping vs release readiness | High | Phases 1–2 only until contributor GA | +| Cloud policy vs local-first conflict | Med | fail-open default; explicit org enroll | + +--- + +## 10. Success metrics + +| Metric | Baseline (now) | Target (post Phase 2) | Target (GA) | +|--------|----------------|------------------------|-------------| +| Agent tool can select plan/explore/general | No (hardcoded explore) | Yes | Yes | +| Explore shell mutation blocked | Partial | 100% allowlist tests | Fuzz green | +| Untrusted project hooks run | Possible | 0 | 0 | +| Time to implement team plugin (skill+mcp+hook) | High friction | <30 min | <10 min marketplace | +| ACP multi-turn with permissions | First-cut | Full load/resume | Editor-certified | +| Subagent resume fidelity | Status only | Full transcript | Full + worktree | +| Doc user-guide chapters | ~0 numbered | 8 | 20+ | + +--- + +## 11. Suggested PR / milestone naming + +```text +feat(contracts): agent spawn DTOs +feat(spawn): wire explore/plan/general through Agent tool +feat(spawn): capability modes + tool filter +feat(spawn): worktree isolation +feat(spawn): resume_from transcripts +refactor(taskruntime): unify background managers +feat(sandbox): sandbox.toml profiles +feat(trust): folder trust store +feat(hooks): pre_tool deny + file/http runners +feat(plugin): multi-component layout +feat(marketplace): multi-source install MVP +feat(tools): monitor + wait + kill +feat(acp): session load/resume +feat(cloud): managed policy apply +docs(user-guide): 01-getting-started … +``` + +--- + +## 12. Resource plan (long horizon) + +| Role | Q1–Q2 | Q3–Q4 | Q5–Q8 | +|------|-------|-------|-------| +| Hawk core (Go) | 1.5 FTE | 1.5 FTE | 1 FTE | +| Security-minded sandbox | 0.5 FTE | 0.5 FTE | 0.25 FTE | +| Community-skills / marketplace content | 0.25 | 0.75 | 0.5 | +| Cloud (TS) | 0 | 0.25 | 0.75 | +| SDK / ACP | 0.25 | 0.25 | 0.75 | +| Docs / DX | 0.25 | 0.5 | 0.5 | + +**Minimum viable long-horizon track** (one engineer): Phases 1→2→3→4 only (~4–5 months) yields most product quality jump. + +--- + +## 13. Appendix A — Call sites to migrate (AgentSpawnFn) + +Verified entry points (non-exhaustive; re-grep at start of Phase 2): + +| File | Usage | +|------|--------| +| `internal/engine/session.go` | field declaration | +| `internal/engine/agent_session_tool.go` | WireAgentTool / spawnSubAgent | +| `internal/engine/stream_tool_exec.go` | ToolContext injection | +| `internal/tool/agent.go` | Agent / MultiAgent | +| `internal/tool/agentic_fetch.go` | research spawn | +| `internal/tool/tool.go` | ToolContext | + +Background: + +| File | System | +|------|--------| +| `internal/tool/background.go` | BackgroundAgentManager | +| `internal/engine/background_runner.go` | BackgroundRunner | +| `internal/engine/agent/background_agent.go` | BackgroundAgentPool | + +--- + +## 14. Appendix B — Mapping Grok names → Hawk names + +| Grok | Hawk | +|------|------| +| `task` tool | `Agent` / `Task` alias | +| `general-purpose` | `general` / `general-purpose` | +| `capability_mode` | same | +| `isolation: worktree` | same | +| `resume_from` | same (replace weak `agent_id` semantics) | +| `sandbox.toml` | `~/.hawk/sandbox.toml` | +| folder trust | `~/.hawk/trusted_folders.toml` | +| `/loop` | `/loop` over CronScheduler | +| plugin marketplace | `hawk plugins` + community registry | +| managed_config | managed policy via hawk-cloud | +| ACP | `hawk acp` | + +--- + +## 15. Appendix C — First 30-day execution checklist + +Week 1 + +- [ ] Land contracts package + tests +- [ ] Document ADR +- [ ] Inventory + delete path for bg managers (design only) + +Week 2 + +- [ ] SpawnRequest internal +- [ ] Agent tool schema expanded (flags behind feature if needed) +- [ ] Wire plan + general (stop hardcoding explore) + +Week 3 + +- [ ] Explore bash allowlist enforcement +- [ ] SpawnResult structured output +- [ ] Integration tests fake LLM + +Week 4 + +- [ ] Worktree isolation MVP +- [ ] Transcript save (resume stub OK) +- [ ] CHANGELOG + user-facing `/help` snippet + +--- + +## 16. Decision log (planning) + +| Decision | Choice | Rationale | +|----------|--------|-----------| +| Language | Go only | Hawk ecosystem is Go | +| First work | Wire existing modes | Types already exist; highest ROI | +| Contracts location | hawk-core-contracts | Engines/SDKs may later honor modes | +| Trust default | On | Security | +| Marketplace before trust? | No | Supply chain | +| Replace yaad? | No | Graph memory superior | +| Replace eyrie? | No | Multi-provider is differentiator | +| Three bg systems | Unify early | Blocks monitor/wait | + +--- + +## 17. Document maintenance + +- Update this plan at each phase exit. +- Mark concept table statuses. +- Link PRs under each phase. +- Supersedes marketing claims in `docs/IMPLEMENTATION-ROADMAP.md` for agent-control scope (that doc remains for marketplace/IDE product ideas). + +--- + +**End of plan.** +Next action if approved: execute **Phase 1 (contracts)** then **Phase 2 PR stack** without waiting for marketplace or cloud. diff --git a/internal/daemon/telegram.go b/internal/daemon/telegram.go index d9ff1361..c69480b4 100644 --- a/internal/daemon/telegram.go +++ b/internal/daemon/telegram.go @@ -227,11 +227,15 @@ func (tg *TelegramGateway) forwardToHawk(ctx context.Context, prompt string) (st defer func() { _ = resp.Body.Close() }() // Limit response body to 1 MiB to prevent memory exhaustion. - body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if err != nil { + slog.Warn("partial read in forwardToHawk response", "error", err) + } var chatResp struct { Response string `json:"response"` } if err := json.Unmarshal(body, &chatResp); err != nil { + // JSON unmarshal may fail with partial body; return raw body as fallback return string(body), nil } return chatResp.Response, nil @@ -255,7 +259,10 @@ func (tg *TelegramGateway) sendMessage(ctx context.Context, chatID int64, text s return err } defer func() { _ = resp.Body.Close() }() - body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096)) + body, err := io.ReadAll(io.LimitReader(resp.Body, 4096)) + if err != nil { + slog.Warn("partial read in sendMessage response", "error", err) + } var result struct { OK bool `json:"ok"` Description string `json:"description"`