Skip to content

plan.md

Latest commit

 

History

History
154 lines (133 loc) · 10.7 KB

plan.md

File metadata and controls

154 lines (133 loc) · 10.7 KB
Raw

Push to Prod — Build Plan

Branch: hackathon/tuffy-cli-submission. Companion doc: HACKATHON_SUBMISSION.md (fill that in as each item below lands — it's the source for the actual Devfolio submission).

Focus, in priority order

  1. Local assistant (tuffy-cli itself) — the base product being submitted.
  2. Elastimem — the own-built memory framework as a whole (~/Projects/elastimem, CodebyKumar/elastimem) is the centerpiece of the bet: temporally-versioned facts, hybrid episodic recall, background extraction/consolidation, the Memory Governor, and the newest addition, the graph layer. Upgrade tuffy-cli to the latest Elastimem and verify the whole framework actually works end-to-end inside tuffy-cli — not just that it imports/builds.
  3. Claude API integration — wire Claude in as the default model, using the credits provided for the event, and verify tool use / Elastimem recall works through Claude specifically. This pairing (Elastimem + Claude, both proven live in tuffy-cli) is the actual submission.
  4. Online/offline mode — a real, enforced runtime toggle (startup selector + /network command) between cloud (Claude, requires network) and fully local (llama.cpp, zero network) operation, with tool availability and the model's own self-description changing accordingly. Demonstrates the local-first story isn't just "a local model exists" — it's a system that knows and honestly reports which mode it's in, and mechanically enforces it.
  5. Stretch, only if 1–4 are solid with time left: Maya Research's native voice models — would replace/augment the current Whisper STT + Piper TTS voice path. Not part of this submission's theme claim (see HACKATHON_SUBMISSION.md → Theme fit) — treat as a bonus demo moment, not a pitch pillar.

Judging rewards a focused proof of the hardest part over many incomplete features — do not start the stretch goal until 1–3 are demo-ready.

Theme claim (locked)

Primary: Next frontier capability (Elastimem as a self-built memory architecture, not a repackaging of vector-DB-plus-chatbot memory — the graph layer is one capability inside it, not the headline). Secondary: Infrastructure everyone else will build on (Elastimem as a standalone library already backing two products). See HACKATHON_SUBMISSION.md for the full reasoning — don't stretch the pitch to cover themes that don't fit.

Why this scope fits the theme

  • Elastimem as a whole — versioned facts, hybrid FTS5+cosine episodic recall, governor-scaled budgets, background consolidation, and now graph-based entity/relationship recall — is a genuine capability gap in most local/terminal agents, not just the graph piece alone. Closest fit to "next frontier capability."
  • Elastimem already exists and is architecturally sound — this is an upgrade-and-verify pass inside the 5-hour window, not a from-scratch build, which is what makes it achievable.
  • Claude as the driving model for both the chat loop and the memory extraction pipeline makes AI "core," not decorative — matches the "AI Depth & Nativeness" judging criterion directly.
  • The online/offline toggle turns "Claude integration" and "local-first" from a marketing claim into an enforced, demoable mechanism: switching modes actually swaps the active model, actually removes network tools from what the model is told it can do, and the model actually reports its own mode correctly when asked — not just a flag that quietly does nothing.

TODO

Elastimem upgrade + full verification

  • Confirm which commit/tag in ~/Projects/elastimem is the latest (includes the graph layer, src/elastimem/graph.py) and whether it's been pushed to CodebyKumar/elastimem on GitHub. Local main was 8 commits ahead of origin — pushed (clean fast-forward) as 58fc1a2.
  • Update tuffy-cli's pyproject.toml git dependency pin for elastimem to that commit/tag. Pinned to 58fc1a2c14c544da0bf9b5a75a142662761d5407; uv sync --extra voice reinstalled.
  • Read src/memory.py (tuffy-cli's Elastimem adapter) and confirm it's using the current Elastimem surface. Calls mem.recall()/mem.remember()/mem.build_context() — but two real gaps found once verification went past "imports cleanly" to "actually reaches the model": src/prompts/templates.py never read the graph_context section build_context() produces (fixed — added a RELATED TOPICS block), and Elastimem's worker_max_tokens default (96) was too small for the combined facts+graph extraction completion, truncating the JSON mid-payload and silently dropping the whole extraction (fixed — raised to 320 in both elastimem.open(...) call sites in src/memory.py).
  • Manually verify the full framework end-to-end in tuffy-cli — for real, not just imports. Recorded a turn naming real entities, drained the worker, confirmed via direct SQLite queries that graph_nodes/graph_edges were actually populated (this is what caught the token-budget bug); then confirmed an entity-naming query populates graph_context and renders as a real "RELATED TOPICS" block in the system prompt (this is what caught the templates.py gap). Also ran Elastimem's own upstream graph test suite (47/47 passing).
  • Run tuffy-cli's existing test suite against the upgraded dependency; fix any breakage. 140/140 passed throughout, including after both fixes.
  • Pulled Elastimem's newer upstream commit (7b56c92, "Raise LITE tier's episodic/graph floor") and re-pinned/reinstalled. graph_hops at LITE is now 1 (was 0) — LITE performs real 1-hop read traversal against whatever graph already exists (writes still require STANDARD/FULL, since those need an LLM extraction call). Re-verified directly against this machine's real LITE tier (~2 GiB available RAM, no ELASTIMEM_TIER override): seed detection and traversal both succeed against the live database. Rendering is still budget-capped at LITE (~10 tokens by default, 25% of LITE's small episodic share) — confirmed a short single-entity result renders correctly, and a larger multi-entity cluster correctly degrades to empty rather than truncating garbled output. Full tuffy-cli suite (141/141) still passes against the upgraded pin.

Claude API integration

  • Build a native Anthropic Messages API provider — src/llm/anthropic_provider.py — not the OpenAI-compatible path (Anthropic's wire format differs: separate endpoint, x-api-key/ anthropic-version headers, top-level system, different SSE event shape). Registered as provider "anthropic" in src/llm/__init__.py / src/models/registry.py.
  • Add Claude model entries — src/models/configs/anthropic.py: Opus 5, Sonnet 5, Haiku 4.5.
  • Group /models output by provider (Local / OpenAI-compatible API / Anthropic).
  • ANTHROPIC_API_KEY added to .env (confirmed gitignored before use).
  • Smoke-test tool use / Elastimem recall specifically against Claude with a real API key. Live-verified: completion + streaming + system-prompt translation; full ReAct tool-use loop (with self-correction) computing 47 × 89 = 4,183 via actual run_python execution; Elastimem facts/episodic recall/build_context() all driven by Claude. Caught and fixed a real bug along the way — the adapter sent both temperature and top_p, which the Messages API rejects with a 400 (fixed by making them mutually exclusive in anthropic_provider.py).
  • Set a Claude model as the default active model. DEFAULT_MODEL in src/models/__init__.py set to claude-haiku-4-5-anthropic (fastest/cheapest Claude, matches "least model of Claude" ask). /models no-arg listing reordered to Local → Anthropic (Claude) → OpenAI-compatible API. Verified via the real load_agent() startup path — loads and completes correctly.

Online/Offline network mode

  • Startup arrow-key selector (select_one() in src/cli/display.py) asks Online or Offline every launch, pre-selected on the last persisted choice (src/settings.py → get_network_mode()/set_network_mode()), defaulting to online on first run. Falls back to plain typed input on non-TTY/Windows (no termios/tty).
  • src/network_mode.py holds the live in-process mode (get_mode()/set_mode()/ is_offline()); main.py reads it at startup to decide the active model, /network online|offline (src/cli/commands.py) changes it mid-session.
  • Offline is a hard restriction, not a hint: main.py refuses to start an API model when offline (falls back to FALLBACK_MODEL if the persisted default isn't local); /models switch refuses to switch to a non-llama_cpp model while offline; /network offline mid-session auto-switches off an active API model to FALLBACK_MODEL.
  • Network-dependent tools (web_search, translate in src/tools/research.py) are gated twice: dropped entirely from the system prompt's tool list while offline (src/tools/registry.py's tool_lines(exclude_names=...), wired from src/prompts/__init__.py) so the model isn't told a tool exists that it can't use, and refused at call time with a same message if the model tries anyway.
  • Identity/self-model (src/identity.py's describe(), src/prompts/templates.py's self_model()) now states current network mode explicitly in every system prompt, so the model never assumes "local" or "online" by default — matches the reworded tuffy persona in personas.yaml.
  • /status now reports live network mode alongside model/provider.

Demo readiness

  • Pick and rehearse one end-to-end scenario that shows graph-memory recall working through Claude, live — this is the centerpiece of the pitch.
  • Second demo beat: show the online/offline toggle live — start online on Claude, flip to /network offline mid-session (watch it fall back to the local model and drop web_search/ translate from the tool list and system prompt), flip back online.
  • Capture a project image/snapshot and record the demo video.
  • Write up sections 3–7 of HACKATHON_SUBMISSION.md from what was actually built.

Stretch (only if time remains after the above)

  • Look at Maya Research's native voice models as a possible swap-in for src/voice/'s current Whisper STT + Piper TTS.
  • Scope a minimal integration (or explicitly decide it's demo-only / not worth the risk this close to the deadline).

Submission logistics

  • Re-run Devfolio MCP lookup once registration/check-in opens (getUserActiveHackathons → getHackathonTracksAndPrizes → getProjectSubmissionGuide).
  • Decide track to apply to once tracks are fetchable.
  • Submit via Devfolio MCP or dashboard before 4:30 PM IST deadline, well before the wire.