Skip to content

CLAUDE.md

Latest commit

 

History

History
148 lines (118 loc) · 8.5 KB

CLAUDE.md

File metadata and controls

148 lines (118 loc) · 8.5 KB
Raw

CLAUDE.md — Testify

Project working notes + history for Claude Code. Read this first every session. One-liner: Audit 100% of regulated phone calls with court-ready, quoted, timestamped evidence.

1. What we're building

A backend that ingests a regulated agent–customer phone call — from a URL or an uploaded audio/video file — and produces one ComplianceScorecard: every requirement in a compliance rule set graded pass / fail / insufficient_evidence, each backed by a verbatim, timestamped, speaker-attributed evidence quote, plus a dollar-exposure risk rollup and a court-ready evidence pack export.

Killer details:

  • Never a fabricated pass. A requirement passes ONLY with a real supporting quote; missing evidence → insufficient_evidence + human review. Dollar exposure only on a fail.
  • Speaker attribution with no ML diarization. Dual-channel call recordings → split channel 0 = agent, channel 1 = customer.
  • Throughput is the ROI. Manual QA samples ~2% of calls; GPU batch-ASR makes 100% coverage economical. Buyer = Chief Compliance Officer / VP of QA at a collections or lending firm (FDCPA / TCPA).

Frontend is built separately; we focus on the backend + API. This product is audio-only — there is NO vision/frames pathway.

2. Hackathon context

Event: AMD Developer Hackathon: ACT II (lablab.ai × AMD × Google DeepMind Gemma).

  • Track 3 — Unicorn (THIS IS US). "Your idea. AMD infrastructure. No benchmarks." Human-judged.
  • T3 judging criteria: (1) Creativity & Originality, (2) Product/Market Potential, (3) Completeness, (4) Use of AMD Platforms.
  • Prizes: T3 = $2,500 / $1,500 / $1,000, + $2,000 "Best AMD-Hosted Gemma Project."
  • Hard requirements: submit on lablab (title, descriptions, tags, cover, video, deck, public GitHub repo w/ README), containerized (Dockerfile + compose), MIT-licensed, runnable.
  • THE AMD requirement (LabLab Admin ruling): demonstrate actual AMD compute usage — run your model on the AMD Jupyter Notebook — shown in repo AND demo. → notebooks/amd_testify_demo.ipynb.
  • Deadline: ~Jul 11, 2026 (exact hour per-user on the lablab "Event Schedule" tab).
  • ⚠️ Never commit secrets. Develop with .env (git-ignored) → scrub → public at submission.

3. Architecture

Heavy perception on the AMD MI300X (the "Use of AMD Platforms" story), reasoning on Gemma.

INPUT: URL or uploaded audio/video
  ├─ 1. INGEST     yt-dlp (or httpx for a direct media link) → the call recording. Node on PATH
  │                lets yt-dlp solve YouTube's JS/PO-token challenge cookie-free.
  ├─ 2. AUDIO      ffmpeg → 16 kHz mono WAV. Dual-channel (stereo) → split ch0=agent, ch1=customer
  │                (ffprobe channel count; pan filter). Mono → one 'unknown' track.
  ├─ 3. ASR        Whisper-large-v3 (HF transformers) per channel  ── MI300X (ROCm) ──
  │                → segments tagged with speaker, merged by start time.
  ├─ 4. GRADE      Gemma (OpenAI-compatible), json_schema structured output. Grades EVERY rule
  │                → ONE ComplianceScorecard (temperature 0). Verbatim quotes only; never invent.
  ├─ 5. VALIDATE   postprocess.normalize(): exposure only on fails, flag insufficient/low-conf for
  │                review, derive overall_verdict; compute_risk() → RiskRollup.
  └─ 6. PERSIST    SQLite job + audit store; FastAPI REST + SSE progress; evidence-pack export.

Pluggable providers (all with graceful fallback + a mock impl):

  • Transcriber: local (transformers Whisper, ROCm/CPU) · mock.
  • Synthesizer: fireworks (Gemma via Fireworks / Google AI Studio shim) · amd (Gemma via local vLLM on MI300X) · mock. Both real profiles share ONE OpenAI-compatible code path.

Key technical facts (don't re-learn)

  • Compliance grading is a PURE-TEXT task (transcript + rule set) → real Gemma, no multimodal workaround. Gemma is text-only on the Google shim, which is exactly right here.
  • Free real Gemma, no card: Google AI Studio. FIREWORKS_BASE_URL= https://generativelanguage.googleapis.com/v1beta/openai/, SYNTH_MODEL=gemma-4-31b-it. (Our synthesizer is endpoint-agnostic; FIREWORKS_* vars just point at Google.)
  • AMD-hosted Gemma ($2k prize): serve Gemma on the MI300X via vLLM (ROCm, OpenAI-compatible), SYNTHESIZER=amd → AMD_LLM_BASE_URL / AMD_LLM_MODEL. scripts/serve_amd_gemma.sh.
  • ROCm PyTorch masquerades as CUDA: torch.cuda.is_available() True on MI300X, device "cuda", torch.version.hip set. Same code CPU/laptop/MI300X. Install torch from the ROCm wheel index only in the cloud/Docker step (not pinned in requirements).
  • Whisper on ROCm: HF transformers Whisper (pure PyTorch). Avoid faster-whisper (CTranslate2 has no ROCm backend).
  • Structured output: response_format=json_schema (+ describe schema in the prompt); fall back to json_object then lenient. _coerce uses raw_decode (Gemini shim appends trailing tokens → "Extra data"). Do NOT proxy through LiteLLM.
  • pass verdict: Python keyword → enum member pass_ with value "pass"; a str Enum serializes to the JSON string "pass" (round-trip verified in tests).

4. Tech stack

  • Python 3.12 + FastAPI + Uvicorn. Pydantic v2 / pydantic-settings.
  • Async in-process job worker (asyncio) + SQLite (stdlib sqlite3) job+audit store.
  • Media: yt-dlp, httpx, ffmpeg (system). ML (GPU, optional): torch/torchaudio (ROCm), transformers, accelerate, soundfile. LLM: openai SDK → Gemma endpoint.
  • Packaging: Dockerfile (slim) + Dockerfile.rocm (MI300X) + compose + Makefile + pytest.

5. API surface

  • POST /api/v1/audits — body { "url": "...", "ruleset"?: [...] } → { job_id } (async).
  • POST /api/v1/audits/upload — multipart audio/video file.
  • GET /api/v1/jobs/{job_id} (+ /events SSE) — status + progress.
  • GET /api/v1/audits/{audit_id} — the ComplianceScorecard.
  • GET /api/v1/audits/{audit_id}/evidence-pack — court-ready evidence pack (JSON).
  • GET /api/v1/audits — list. GET /health · GET /api/v1/meta (providers / GPU).

6. Rule set

Default FDCPA/TCPA starter template in fixtures/default_ruleset.json (~6 rules: mini_miranda, caller_identification, no_threats_or_harassment, no_false_representations, right_party_verification, calling_time_window). Requests may override via the ruleset body field. Illustrative — not legal advice.

7. Status / history log

  • 2026-07-11 — Testify built by transforming the RecipeReel scaffold. Removed the entire vision/frames pathway (audio-only). New content model app/models/audit.py (ComplianceScorecard + Verdict.pass_/fail/insufficient_evidence, Speaker, Severity, OverallVerdict, RiskRollup, EvidenceQuote; to_evidence_pack() / to_evidence_html()). Pipeline: ingesting → extracting_audio (dual-channel split) → transcribing (per-channel Whisper, speaker-tagged) → synthesizing (Gemma grades the rule set) → validating (risk rollup + verdicts). Config: TranscriberKind local|mock, SynthesizerKind fireworks|amd|mock, ruleset_path, max_audio_seconds; removed all vision settings. API renamed recipes→audits, recipe_id→audit_id, added /evidence-pack; optional per-request ruleset override wired through the job. Fixtures: default_ruleset.json, dual-channel sample_transcript.json, sample_scorecard.json (1 pass, 1 fail critical/$1500, 2 insufficient). Docker/compose/Makefile/.env.example/README/docs/notebook all Testify. Verified: import app.main clean, pytest 13 green (incl. pass round-trip + risk-rollup math), ruff clean, mock smoke test (POST→poll→scorecard w/ pass+fail+insufficient
    • non-zero exposure→evidence-pack w/ disclaimer). TODO (user): run the notebook on the MI300X pod + record demo + export deck→PDF + submit. Do NOT submit the lablab form or change the team.

8. Conventions for Claude

  • Keep the pipeline always runnable: any provider missing → fall back, never hard-crash. MOCK_MODE=true runs the whole pipeline offline from fixtures with zero extra deps.
  • Never commit secrets. Config via env / .env (git-ignored). .env.example documents keys.
  • Match existing code style; keep modules small and single-purpose.
  • When you finish a meaningful chunk, append a dated line to the Status log above.
  • Optimize copy for the T3 criteria (esp. "Use of AMD Platforms" throughput story + product framing).