feat(notes): local-first voice memo transcription into notes (ODIAA-2331)

CHANGELOG.md

@@ -55,6 +55,18 @@ First release of the fork on npm as `@odience-network/thunderbird-cli-enhanced`.
   actively maintained) and sanitizes with
   [`sanitize-html`](https://www.npmjs.com/package/sanitize-html) (parser-driven, not regex-based)
   before the HTML ever reaches an email draft.
+- **`tb notes transcribe <audioFile>`** / MCP `note_transcribe` — voice memo transcription into
+  Notes. Local-first: at runtime it detects a locally-installed speech-to-text engine —
+  [whisper.cpp](https://github.com/ggml-org/whisper.cpp) (`whisper-cli` on PATH, or
+  `TB_WHISPER_CPP_BIN`; needs a model via `--model`/`TB_WHISPER_CPP_MODEL`) or
+  [faster-whisper](https://github.com/SYSTRAN/faster-whisper)'s `whisper-ctranslate2` CLI wrapper
+  (on PATH, or `TB_FASTER_WHISPER_BIN`) — and shells out to it; no audio or text ever leaves the
+  machine, and no API key is read. Absence of either engine is a clean `NO_ENGINE` error with
+  install hints, not a stack trace. The transcript is saved as a note with the source audio
+  filename and engine used recorded in front matter (`source`, `engine`). No cloud STT provider
+  is implemented — that's a separate vendor/stack decision requiring its own sign-off before any
+  network path ships (see the ODIAA-2331 PR for the option A vs B trade-off). Tested against a
+  stubbed engine (`test/fixtures/stub-*.mjs`).
 - `--from <identityId>` on `tb forward` (`from` on MCP `email_forward`); `--from` on reply/forward may now name an identity from any account (unknown identities are still rejected).
 - `--subject <text>` on `tb reply` (`subject` on MCP `email_reply`).
 - `npm run test:draft` — draft-routing regression suite (identity resolution, cross-account `--from`, Gmail dual-drafts warning, unconfirmed-save errors, reply subject/quote/HTML handling), also part of `test:extension`.

SPEC.md

@@ -21,7 +21,7 @@
 │       ┌────────────────────┬──────────────────────┐         │
 │       ↕                    ↕                      ↕         │
 │  tb CLI (Node)      tb-mcp Server          Direct HTTP      │
-│  (47 commands)      (31 MCP tools)         (curl, scripts)  │
+│  (75 commands)      (42 MCP tools)         (curl, scripts)  │
 │       ↕                    ↕                                │
 │  AI Agent           Claude Desktop                          │
 │  (Claude Code)      (stdio MCP transport)                   │
@@ -40,8 +40,8 @@
 | **Thunderbird** | Host | Source of truth. Stores all emails, syncs IMAP, renders UI for human oversight |
 | **Extension** (background.js) | Inside Thunderbird | Pure WebExtension. Connects to bridge via WebSocket. Checks every request against the build-time access policy (`access-control.js`, see [docs/ACCESS-CONTROL.md](docs/ACCESS-CONTROL.md)), then translates it into `messenger.*` API calls |
 | **Bridge** (bridge.js) | Host (daemon) | Stateless HTTP↔WebSocket proxy. Receives HTTP from CLI/MCP, forwards to extension, returns response. No business logic |
-| **CLI** (tb) | Host or Docker | Thin HTTP client. Parses args, calls bridge, outputs JSON to stdout. 47 commands. Auto-starts bridge daemon if not running. Zero state |
-| **MCP Server** (tb-mcp) | Host (alongside Claude Desktop) | Stdio-based MCP server. Exposes 41 curated tools to Claude Desktop and other MCP clients. Auto-starts bridge daemon if not running. Reuses CLI's HTTP client to call bridge |
+| **CLI** (tb) | Host or Docker | Thin HTTP client. Parses args, calls bridge, outputs JSON to stdout. 75 commands. Auto-starts bridge daemon if not running. Zero state |
+| **MCP Server** (tb-mcp) | Host (alongside Claude Desktop) | Stdio-based MCP server. Exposes 42 curated tools to Claude Desktop and other MCP clients. Auto-starts bridge daemon if not running. Reuses CLI's HTTP client to call bridge |
 
 ### Request flow
 
@@ -580,6 +580,12 @@ tb notes delete <name> --confirm
 tb notes search <query>
 # Returns: [{name, title, snippet}]
 
+# Transcribe a local audio file to text with a local speech-to-text engine
+# and save the transcript as a note. Local-first: no audio or text ever
+# leaves this machine.
+tb notes transcribe <audioFile> [--engine whisper-cpp|faster-whisper] [--model <path>]
+  [--language <code>] [--save-as <name>] [--title <t>] [--append]
+
 # Render a note's Markdown to sanitized HTML and open it as an email draft.
 # Uses the same /compose route as `tb compose` — never sends.
 tb notes to-draft <name> --to <address> [--cc <a>] [--bcc <a>] [--subject <t>] [--from <identityId>] [--open]
@@ -589,6 +595,18 @@ Markdown → HTML uses `marked` (rendering) and `sanitize-html` (stripping
 scripts, event handlers, and unsafe URL schemes) before the HTML ever
 reaches a compose draft.
 
+`tb notes transcribe` detects a locally-installed speech-to-text engine at
+runtime — [whisper.cpp](https://github.com/ggml-org/whisper.cpp) (`whisper-cli`
+on PATH, or `TB_WHISPER_CPP_BIN`; requires a model file via `--model` or
+`TB_WHISPER_CPP_MODEL`) or [faster-whisper](https://github.com/SYSTRAN/faster-whisper)'s
+`whisper-ctranslate2` CLI wrapper (on PATH, or `TB_FASTER_WHISPER_BIN`) — and
+shells out to it. If neither is found it fails with a clean error and install
+hints rather than a stack trace. No cloud STT provider is implemented: that's
+a separate vendor/stack decision (see the ODIAA-2331 PR for the option A vs B
+trade-off) and would need its own sign-off before any network path ships.
+The engine used and the source audio filename are recorded in the saved
+note's front matter (`engine`, `source`).
+
 ### 19. Calendars and calendar events
 
 Experimental — requires the `calendar` Experiment API vendored into the extension
@@ -1317,7 +1335,7 @@ The MCP server:
 
 ### Tool Catalog
 
-The 41 MCP tools are **curated** for AI agent use cases. Bulk admin operations (folder CRUD, identity management, bulk delete, etc.) are intentionally excluded — they belong in the CLI for explicit human control.
+The 42 MCP tools are **curated** for AI agent use cases. Bulk admin operations (folder CRUD, identity management, bulk delete, etc.) are intentionally excluded — they belong in the CLI for explicit human control.
 
 | MCP Tool | Maps to CLI commands |
 |----------|---------------------|
@@ -1339,6 +1357,7 @@ The 41 MCP tools are **curated** for AI agent use cases. Bulk admin operations (
 | `note_read` | `tb notes read` ("Use as Context") |
 | `note_save` | `tb notes save` ("Save to Notes") |
 | `note_append` | `tb notes append` |
+| `note_transcribe` | `tb notes transcribe` (local speech-to-text engine, auto-detected) |
 | `note_to_draft` | `tb notes to-draft` (draft/open modes, never sends) |
 | `contact_search` | `tb contacts`, `tb contacts-search` |
 | `contact_create` | `tb contacts create` (requires `contactsWrite`) |
@@ -1370,7 +1389,7 @@ local notes workspace directly.
 
 ### Why fewer MCP tools than CLI commands?
 
-| | CLI (47 commands) | MCP (31 tools) |
+| | CLI (75 commands) | MCP (42 tools) |
 |---|---|---|
 | Audience | Humans + scripts | AI agents |
 | Discovery | `tb --help` | Tool descriptions in LLM context |

cli/src/cli.js

@@ -10,7 +10,7 @@
 import { createRequire } from "module";
 import { Command } from "commander";
 import { api, output, outputError, getConfig, parseRelativeDate } from "./client.js";
-import { extname } from "path";
+import { basename, extname } from "path";
 import {
   listNotes,
   readNote,
@@ -20,6 +20,7 @@ import {
   searchNotes,
   renderNoteHtml,
 } from "./notes.js";
+import { transcribeAudio } from "./stt.js";
 import {
   startOfDay,
   addDays,
@@ -1495,6 +1496,33 @@ notes
     output(searchNotes(query), getFormat(g), getOutputOpts(g));
   }));
 
+notes
+  .command("transcribe <audioFile>")
+  .description(
+    "Transcribe a local audio file to text with a local speech-to-text engine and save it as a note " +
+      "(no audio ever leaves this machine)"
+  )
+  .option("--engine <id>", "STT engine to use: whisper-cpp or faster-whisper (default: auto-detect)")
+  .option("--model <path>", "model file path (required for whisper.cpp; optional override for faster-whisper)")
+  .option("--language <code>", "spoken language hint, e.g. en")
+  .option("--save-as <name>", "note name to save as (defaults to the audio file's base name)")
+  .option("--title <text>", "note title (defaults to the note name)")
+  .option("--append", "append to an existing note instead of overwriting")
+  .action(run(async (audioFile, opts) => {
+    const g = program.opts();
+    const { text, engine } = transcribeAudio(audioFile, {
+      engine: opts.engine,
+      model: opts.model,
+      language: opts.language,
+    });
+    const name = opts.saveAs || basename(audioFile, extname(audioFile));
+    const source = `voice-memo:${basename(audioFile)}`;
+    const result = opts.append
+      ? appendNote(name, text, { title: opts.title, source, engine })
+      : saveNote(name, text, { title: opts.title, source, engine });
+    output({ ...result, engine, transcript: text }, getFormat(g), getOutputOpts(g));
+  }));
+
 notes
   .command("to-draft <name>")
   .description("Render a note to an HTML email draft (never sends)")

cli/src/notes.js

@@ -137,6 +137,7 @@ export function listNotes(opts = {}) {
         title: meta.title || name,
         created: meta.created || stat.birthtime.toISOString(),
         source: meta.source ?? null,
+        engine: meta.engine ?? null,
         size: stat.size,
         modified: stat.mtime.toISOString(),
       };
@@ -158,6 +159,7 @@ export function readNote(name, opts = {}) {
     title: meta.title || safeName,
     created: meta.created || stat.birthtime.toISOString(),
     source: meta.source ?? null,
+    engine: meta.engine ?? null,
     modified: stat.mtime.toISOString(),
     body: body.trimEnd(),
   };
@@ -171,6 +173,7 @@ export function saveNote(name, body, opts = {}) {
     title: opts.title || safeName,
     created: new Date().toISOString(),
     source: opts.source ?? undefined,
+    engine: opts.engine ?? undefined,
   };
   writeFileSync(filePath, serializeFrontMatter(meta) + String(body ?? "").trimEnd() + "\n", "utf-8");
   return { name: safeName, title: meta.title, path: filePath };
@@ -186,12 +189,18 @@ export function appendNote(name, body, opts = {}) {
     const nextMeta = { ...meta };
     if (opts.title) nextMeta.title = opts.title;
     if (opts.source !== undefined) nextMeta.source = opts.source;
+    if (opts.engine !== undefined) nextMeta.engine = opts.engine;
     const combined = `${existingBody.trimEnd()}\n\n${String(body ?? "").trimEnd()}\n`;
     writeFileSync(filePath, serializeFrontMatter(nextMeta) + combined, "utf-8");
     return { name: safeName, title: nextMeta.title || safeName, path: filePath, created: false };
   }
 
-  const meta = { title: opts.title || safeName, created: new Date().toISOString(), source: opts.source ?? undefined };
+  const meta = {
+    title: opts.title || safeName,
+    created: new Date().toISOString(),
+    source: opts.source ?? undefined,
+    engine: opts.engine ?? undefined,
+  };
   writeFileSync(filePath, serializeFrontMatter(meta) + String(body ?? "").trimEnd() + "\n", "utf-8");
   return { name: safeName, title: meta.title, path: filePath, created: true };
 }

cli/src/stt.js

@@ -0,0 +1,124 @@
+/**
+ * Local speech-to-text for voice memo transcription.
+ *
+ * Local-first by design: every engine here shells out to a binary the user
+ * installs themselves and runs entirely on their machine. No audio or text
+ * ever leaves the machine, no network call is made, and no API key is read
+ * or required. A cloud STT provider is a separate vendor/stack decision and
+ * is intentionally not implemented here (see the ODIAA-2331 PR description
+ * for the option A vs B trade-off).
+ *
+ * Engines are detected at runtime (env var override, then well-known PATH
+ * binary names) so the CLI works whether the user has whisper.cpp,
+ * faster-whisper, both, or neither installed — absence produces a clean
+ * error with an install hint rather than a stack trace.
+ */
+
+import { existsSync, mkdtempSync, readFileSync, rmSync } from "fs";
+import { execFileSync } from "child_process";
+import { tmpdir } from "os";
+import { basename, delimiter, extname, join, resolve } from "path";
+
+function invalidArgs(message) {
+  return Object.assign(new Error(`INVALID_ARGS: ${message}`), { code: "INVALID_ARGS" });
+}
+
+// Default cap on how long an engine may run before we give up and kill it —
+// guards against a hung or runaway process rather than any expectation of
+// how long real transcription takes.
+const DEFAULT_TIMEOUT_MS = 10 * 60 * 1000;
+
+function which(names) {
+  const dirs = (process.env.PATH || "").split(delimiter);
+  for (const dir of dirs) {
+    if (!dir) continue;
+    for (const name of names) {
+      const candidate = join(dir, name);
+      if (existsSync(candidate)) return candidate;
+    }
+  }
+  return null;
+}
+
+const ENGINES = [
+  {
+    id: "whisper-cpp",
+    label: "whisper.cpp",
+    resolveBin: () => process.env.TB_WHISPER_CPP_BIN || which(["whisper-cli", "whisper-cpp"]),
+    installHint:
+      "whisper.cpp: install it (https://github.com/ggml-org/whisper.cpp), put `whisper-cli` on PATH " +
+      "or set TB_WHISPER_CPP_BIN to its path, and pass --model (or set TB_WHISPER_CPP_MODEL) to a " +
+      "downloaded ggml model file.",
+    transcribe(bin, audioFile, outDir, opts) {
+      const model = opts.model || process.env.TB_WHISPER_CPP_MODEL;
+      if (!model) throw invalidArgs("whisper.cpp requires a model file: pass --model or set TB_WHISPER_CPP_MODEL");
+      if (!existsSync(model)) throw invalidArgs(`model file not found: ${model}`);
+      const outPrefix = join(outDir, "transcript");
+      const args = ["-m", model, "-f", audioFile, "-otxt", "-of", outPrefix, "-nt"];
+      if (opts.language) args.push("-l", opts.language);
+      execFileSync(bin, args, { stdio: ["ignore", "pipe", "pipe"], timeout: opts.timeoutMs ?? DEFAULT_TIMEOUT_MS });
+      return readFileSync(`${outPrefix}.txt`, "utf-8");
+    },
+  },
+  {
+    id: "faster-whisper",
+    label: "faster-whisper",
+    resolveBin: () => process.env.TB_FASTER_WHISPER_BIN || which(["whisper-ctranslate2"]),
+    installHint:
+      "faster-whisper: `pip install whisper-ctranslate2` (its CLI wrapper), ensure `whisper-ctranslate2` " +
+      "is on PATH, or set TB_FASTER_WHISPER_BIN to its path.",
+    transcribe(bin, audioFile, outDir, opts) {
+      const args = [audioFile, "--output_format", "txt", "--output_dir", outDir];
+      if (opts.model) args.push("--model", opts.model);
+      if (opts.language) args.push("--language", opts.language);
+      execFileSync(bin, args, { stdio: ["ignore", "pipe", "pipe"], timeout: opts.timeoutMs ?? DEFAULT_TIMEOUT_MS });
+      const base = basename(audioFile, extname(audioFile));
+      return readFileSync(join(outDir, `${base}.txt`), "utf-8");
+    },
+  },
+];
+
+export function listEngines() {
+  return ENGINES.map((e) => ({ id: e.id, label: e.label, available: Boolean(e.resolveBin()) }));
+}
+
+function resolveEngine(preferredId) {
+  const candidates = preferredId ? ENGINES.filter((e) => e.id === preferredId) : ENGINES;
+  if (preferredId && candidates.length === 0) {
+    throw invalidArgs(`unknown engine '${preferredId}'; expected one of: ${ENGINES.map((e) => e.id).join(", ")}`);
+  }
+  for (const engine of candidates) {
+    const bin = engine.resolveBin();
+    if (bin) return { engine, bin };
+  }
+  const hints = candidates.map((e) => `  - ${e.installHint}`).join("\n");
+  throw Object.assign(
+    new Error(
+      "NO_ENGINE: no local speech-to-text engine found.\n" +
+        `${hints}\n` +
+        "Cloud speech-to-text is not supported here (local-first by design) — no audio ever leaves this machine."
+    ),
+    { code: "NO_ENGINE" }
+  );
+}
+
+export function transcribeAudio(audioFile, opts = {}) {
+  if (typeof audioFile !== "string" || !audioFile.trim()) throw invalidArgs("audio file path is required");
+  const resolvedAudio = resolve(audioFile);
+  if (!existsSync(resolvedAudio)) {
+    throw Object.assign(new Error(`NOT_FOUND: audio file '${audioFile}' does not exist`), { code: "NOT_FOUND" });
+  }
+
+  const { engine, bin } = resolveEngine(opts.engine);
+  const outDir = mkdtempSync(join(tmpdir(), "tb-transcribe-"));
+  try {
+    const text = engine.transcribe(bin, resolvedAudio, outDir, opts).trim();
+    return { text, engine: engine.id, audioFile: resolvedAudio };
+  } catch (err) {
+    if (err.code === "INVALID_ARGS" || err.code === "NOT_FOUND") throw err;
+    const detail = err.stderr ? String(err.stderr).slice(0, 2000).trim() : err.message;
+    throw Object.assign(new Error(`ENGINE_FAILED: ${engine.label} failed: ${detail}`), { code: "ENGINE_FAILED" });
+  } finally {
+    rmSync(outDir, { recursive: true, force: true });
+  }
+}

docs/COMMANDS.md

@@ -271,6 +271,9 @@ tb notes append <name> --body "more text" [--body-file <path>] [--title <t>] [--
                                                        # append, creates if missing
 tb notes delete <name> --confirm                      # delete a note
 tb notes search <query>                               # search titles and bodies
+tb notes transcribe <audioFile> [--engine whisper-cpp|faster-whisper] [--model <path>]
+  [--language <code>] [--save-as <name>] [--title <t>] [--append]
+                                                       # local speech-to-text, save transcript as a note
 tb notes to-draft <name> --to <address> [--cc <a>] [--bcc <a>] [--subject <t>] [--from <identityId>] [--open]
                                                        # render Markdown to sanitized HTML, open as draft (never sends)
 ```

mcp/src/notes.js

@@ -140,6 +140,7 @@ export function listNotes(opts = {}) {
         title: meta.title || name,
         created: meta.created || stat.birthtime.toISOString(),
         source: meta.source ?? null,
+        engine: meta.engine ?? null,
         size: stat.size,
         modified: stat.mtime.toISOString(),
       };
@@ -161,6 +162,7 @@ export function readNote(name, opts = {}) {
     title: meta.title || safeName,
     created: meta.created || stat.birthtime.toISOString(),
     source: meta.source ?? null,
+    engine: meta.engine ?? null,
     modified: stat.mtime.toISOString(),
     body: body.trimEnd(),
   };
@@ -174,6 +176,7 @@ export function saveNote(name, body, opts = {}) {
     title: opts.title || safeName,
     created: new Date().toISOString(),
     source: opts.source ?? undefined,
+    engine: opts.engine ?? undefined,
   };
   writeFileSync(filePath, serializeFrontMatter(meta) + String(body ?? "").trimEnd() + "\n", "utf-8");
   return { name: safeName, title: meta.title, path: filePath };
@@ -189,12 +192,18 @@ export function appendNote(name, body, opts = {}) {
     const nextMeta = { ...meta };
     if (opts.title) nextMeta.title = opts.title;
     if (opts.source !== undefined) nextMeta.source = opts.source;
+    if (opts.engine !== undefined) nextMeta.engine = opts.engine;
     const combined = `${existingBody.trimEnd()}\n\n${String(body ?? "").trimEnd()}\n`;
     writeFileSync(filePath, serializeFrontMatter(nextMeta) + combined, "utf-8");
     return { name: safeName, title: nextMeta.title || safeName, path: filePath, created: false };
   }
 
-  const meta = { title: opts.title || safeName, created: new Date().toISOString(), source: opts.source ?? undefined };
+  const meta = {
+    title: opts.title || safeName,
+    created: new Date().toISOString(),
+    source: opts.source ?? undefined,
+    engine: opts.engine ?? undefined,
+  };
   writeFileSync(filePath, serializeFrontMatter(meta) + String(body ?? "").trimEnd() + "\n", "utf-8");
   return { name: safeName, title: meta.title, path: filePath, created: true };
 }