1616- **2026-04-23 (session 12)** — 2 parts. **(a) Competitive scan: SkillClaw (AMAP-ML, arXiv 2604.08377)** reviewed. Cloned at `_external/SkillClaw/` (gitignored). Complementary, not a scoop. Borrow-list parked in memory `skillclaw_competitor.md` + §0.6. **(b) Phase 4 paper-replication REAL RUN executed multiple times (~$0.30 total, Gemini 2.5 Pro).** Discovered + fixed 4 critical infra gaps the scaffolding had silently left: (1) `SpreadsheetEvaluator` couldn't handle range (`J2:J4`) or quoted-sheet (`'COLLECTION'!A1`) answer_positions — fixed with range tuple compare + quote stripping; (2) `AnalystDispatcher` was wired to simple `ErrorAnalyst` not the agentic one — flipped to `AgenticErrorAnalyst` (the USP); (3) `_ground_truth_text` couldn't read xlsx (tried UTF-8 decode on binary) — added `_dump_xlsx_as_text` helper so `read_ground_truth` tool actually works for spreadsheet domain; (4) pydantic's `TypeAdapter.json_schema()` emits `$defs` + `$ref` which Gemini OpenAI-compat rejects when embedded inside tool params — added `_inline_refs` resolver. **291 unit tests passing** (+6: 5 evaluator range/quoted-sheet, 2 agentic xlsx GT). Proof-of-life confirmed: pipeline runs end-to-end agentic with real xlsx GT visible; quality gate (`drop`) fires correctly; merger produces 0-1 ops at n_evolve=5 (prevalence threshold ≥2 too tight for subset this small — paper-level delta needs n_evolve ≥ 15-20, deferred). Committed `bb66c7d`.
1717- **2026-04-24 (session 13)** — **Phase 5: LangChain harness + evidence adapters.** Two pieces shipped. **(a)** `LangChainHarnessAdapter` at `trace2skill/harnesses/langchain.py` (~200 LOC) — wraps LangChain's `AgentExecutor`, per-task executor build with skill-in-system-prompt + `create_tool_calling_agent`, converts `intermediate_steps` → `ReActStep`. Lazy langchain import. 13 unit tests via mock executor. **(b)** `LangChainEvidenceAdapter` at `trace2skill/evidence_adapters/langchain.py` (~280 LOC) — backed by LangSmith API. `collect(run_id)` fetches the root run + child runs, walks all descendants of `run_type == "tool"` (including nested sub-chains), orders by start_time, extracts feedback via `client.list_feedback`. `behavioral` has duration + token counts + project; `execution` has tool call/error counts with capped error snippets; feedback API errors swallowed gracefully (returns None) so evidence collection never fails on a flaky side-channel. 21 unit tests with `_FakeRun`/`_FakeFeedback` duck types + `patch("langsmith.Client")` for lazy-ctor verification. Added `[langchain]` pyproject extra (covers both adapters; langsmith comes transitively with langchain). Both modules pass `ruff` clean + `pyright --strict` 0 errors. **319 unit tests passing** (+21 from 298). Committed `2c23518` + `41a4e69`. Remaining Phase 5: Cline / OpenCode harness adapters, Gemini native SDK (thinking-mode), `examples/04_langchain_rag_agent/` demo, LangSmith thread-id convention for `next_user_turns`.
1818- ** 2026-04-24 (session 14 slice 1)** — ** Phase 7 kickoff: project polish.** README rewrite (was stale "Phase 0 scaffolding" — totally misleading while framework is at Phase 5/6 with 319 tests, 3 harness adapters, 5 provider families, 2 evidence sources). New README has real quickstart (example 02), minimal Python API example, architecture diagram, adapter matrix, honest status ("pre-alpha, actively developed"), citation. CONTRIBUTING.md added (closes Phase 0 deferred item): dev setup, 3-check gate (pytest + ruff + pyright strict), conventions, "add a new adapter" walkthrough with file pointers. ` .github/ISSUE_TEMPLATE/ ` bug + feature templates, plus ` pull_request_template.md ` with plan.md phase + checklist. Committed ` 1e6d99b ` .
19- - ** 2026-04-24 (session 14 slice 2)** — ** Phase 7 docs site.** mkdocs-material configured at ` mkdocs.yml ` with indigo palette + instant navigation + code-copy + pymdownx extensions. 10 doc pages under ` docs/ ` : ` index ` (landing + ASCII pipeline diagram), ` getting-started ` (install + Windows-path caveat + Example 02 walkthrough), ` architecture ` (4 axes + 3 stages + data model + async choices), ` harnesses/{index,claude-code,langchain,custom} ` (custom.md has a ~ 30-line skeleton new adapter can start from), ` providers ` , ` evidence ` , ` rubrics ` (no-regex rule + bilingual pattern), ` roadmap ` (summary pointing at plan.md), ` contributing ` (summary pointing at CONTRIBUTING.md). Added ` [docs] ` pyproject extra + ` .github/workflows/docs.yml ` that builds on push/PR and deploys to GitHub Pages on main. ` mkdocs build --strict ` passes. ` /site/ ` gitignored. Remaining slice 3: PyPI publish prep.
19+ - ** 2026-04-24 (session 14 slice 2)** — ** Phase 7 docs site.** mkdocs-material configured at ` mkdocs.yml ` with indigo palette + instant navigation + code-copy + pymdownx extensions. 10 doc pages under ` docs/ ` : ` index ` (landing + ASCII pipeline diagram), ` getting-started ` (install + Windows-path caveat + Example 02 walkthrough), ` architecture ` (4 axes + 3 stages + data model + async choices), ` harnesses/{index,claude-code,langchain,custom} ` (custom.md has a ~ 30-line skeleton new adapter can start from), ` providers ` , ` evidence ` , ` rubrics ` (no-regex rule + bilingual pattern), ` roadmap ` (summary pointing at plan.md), ` contributing ` (summary pointing at CONTRIBUTING.md). Added ` [docs] ` pyproject extra + ` .github/workflows/docs.yml ` that builds on push/PR and deploys to GitHub Pages on main. ` mkdocs build --strict ` passes. ` /site/ ` gitignored. Committed ` daba526 ` .
20+ - **2026-04-24 (session 14 slice 3)** — **Phase 7 PyPI publish prep.** Version bumped `0.0.1` → `0.1.0` (reflects the real functional surface: 3 harness adapters + 2 evidence adapters + 5 provider families + 319 tests, not "pre-alpha scaffolding"). Dev Status classifier upgraded Pre-Alpha → Alpha. Classifiers expanded: `Intended Audience :: Science/Research`, `Operating System :: OS Independent`, `Python :: 3` / `3.13`, `Topic :: Software Development :: Libraries :: Python Modules`, `Typing :: Typed`. Keywords broadened (`claude-code`, `langchain`, `react`, `prompt-engineering`, `agentic`). `[project.urls]` expanded from 2 → 6 (Homepage, Documentation→gh-pages, Repository, Issues, Changelog, Paper). `trace2skill/py.typed` marker added so downstream users get type info without `reportMissingTypeStubs` noise. `[tool.hatch.build]` explicit include/exclude lists to keep `tests/`, `benchmarks/`, `_external/`, `examples/`, `docs/`, `site/` OUT of the wheel. `CHANGELOG.md` written in Keep-a-Changelog format. `python -m build` produces clean `trace2skill-0.1.0-py3-none-any.whl` (101KB) + sdist (75KB) with prompts + py.typed + LICENSE + entry_points.txt shipped. Wheel METADATA inspected — all classifiers + URLs + keywords propagate correctly. **Not uploaded to PyPI yet** (needs user's PyPI account + `twine upload`). 319 tests still pass.
2021
2122---
2223
@@ -44,9 +45,10 @@ Không đụng vào weights của model (không phải finetuning). Không onlin
4445
4546## 0.5. Progress Snapshot
4647
47- > ** Last updated:** 2026-04-24 (session 14 slices 1+2 — Phase 7 project polish + mkdocs site)
48- > ** Sessions invested:** 14 (Phase 0 → 3.5 → 6 → 5 subset → 4 scaffolding → SkillClaw + Phase 4 infra → Phase 5 LangChain x2 → Phase 7 project polish + docs)
49- > ** Test status:** 319 unit + e2e + 7 live integration passing (unchanged — session 14 is docs-only). ` ruff ` clean, ` pyright --strict ` 0 errors trên source em viết. Pre-existing 1 Windows path test failure (` test_slug_matches_observed_claude_cli_rule ` — hardcoded Unix path).
48+ > ** Last updated:** 2026-04-24 (session 14 slices 1+2+3 — Phase 7 polish + docs + PyPI prep)
49+ > ** Sessions invested:** 14 (Phase 0 → 3.5 → 6 → 5 subset → 4 scaffolding → SkillClaw + Phase 4 infra → Phase 5 LangChain x2 → Phase 7 polish + docs + PyPI prep)
50+ > ** Test status:** 319 unit + e2e + 7 live integration passing (unchanged — session 14 is docs + packaging, no code). ` ruff ` clean, ` pyright --strict ` 0 errors trên source em viết. Pre-existing 1 Windows path test failure.
51+ > ** Version:** 0.1.0 (bumped session 14 slice 3). Wheel + sdist build clean with ` python -m build ` . PyPI upload pending user's account.
5052> ** Framework state:** ** plug-and-forget với real Claude Code + multi-provider + paper-replication infra complete (session 12).** Agentic error analyst with xlsx-aware GT now actually active in pipeline (was silently wired to simple analyst before). Paper-replication runs end-to-end on real SpreadsheetBench data with Gemini Pro 2.5; signal-level delta paper-faithful run (n_evolve ≥ 15-20) deferred for cost/time.
5153> ** Competitive landscape:** SkillClaw (arXiv 2604.08377) reviewed session 12. Complementary product. See §0.6 Competitive landscape + memory ` skillclaw_competitor.md ` .
5254
@@ -62,7 +64,7 @@ Không đụng vào weights của model (không phải finetuning). Không onlin
6264| 4. Paper Replication | 🟢 Infra complete (session 12) | Subset scaffolding (session 11) + 4 infra fixes (session 12): SpreadsheetEvaluator range + quoted-sheet; AnalystDispatcher wired to AgenticErrorAnalyst (not simple); ` _ground_truth_text ` xlsx-aware dump; ` _inline_refs ` schema resolver for Gemini tool-calling. Real runs executed with Gemini 2.5 Pro (~ $0.30 total session 12). End-to-end functional including agentic analyst quality gate firing. Paper-level delta (n_evolve ≥ 15-20, ~ $0.30-0.40, ~ 20 min wall clock) deferred. Full paper scale (6 conditions × 2 models × 200/200) deferred. |
6365| 5. Multi-harness Expansion | 🟡 Providers + LangChain pair done (session 10 + 13) | ✅ ` OpenAICompatibleProvider ` generic (session 10). ✅ ` LangChainHarnessAdapter ` + ✅ ` LangChainEvidenceAdapter ` via LangSmith (session 13) — 34 unit tests. Remaining: Cline/OpenCode harness, Gemini native SDK (if thinking-mode needed), LangChain example demo. |
6466| 6. Semi-online Closed Loop | ✅ Done (MVP) | SessionStore (SQLite), ` evolve_from_trajectories() ` skip-Stage-1, skill deploy + atomic backup + rollback, ` trace2skill evolve-online ` + ` trace2skill rollback ` CLI, example 03 với SessionEnd hook. Skip y=0 trajectories (agentic ErrorAnalyst cần GT — paper §2.3, heuristic error mode defer). |
65- | 7. Release & Ecosystem | 🟡 Slices 1+2 done (session 14) | ✅ Slice 1: README rewrite + CONTRIBUTING.md + ` .github/ISSUE_TEMPLATE/ ` + PR template. ✅ Slice 2: mkdocs-material docs site (8 pages: index / getting-started / architecture / harnesses x3 / providers / evidence / rubrics / roadmap / contributing) + ` mkdocs.yml ` + ` [docs] ` pyproject extra + ` .github/workflows/docs.yml ` for gh-pages deploy on main. ` mkdocs build --strict ` passes. Remaining slice 3: PyPI publish prep (metadata + version bump + wheel test ), launch materials. |
67+ | 7. Release & Ecosystem | 🟢 Slices 1+2+3 done (session 14) | ✅ Slice 1 project polish. ✅ Slice 2 mkdocs docs site. ✅ Slice 3 PyPI publish prep: version ` 0.0.1 ` → ` 0.1.0 ` , classifiers expanded (Alpha + Science/Research + Python 3.11/3.12/3.13 + Typing::Typed), keywords broadened, ` [project.urls] ` expanded to 6 links (Docs + Repo + Issues + Changelog + Paper), ` trace2skill/py.typed ` marker added, ` hatch.build ` include/exclude explicit, ` CHANGELOG.md ` written. ` python -m build` produces clean ` trace2skill-0.1.0-py3-none-any.whl ` (101KB) + ` .tar.gz ` (75KB) with prompts + py.typed + LICENSE + entry_points shipped. Not pushed to PyPI — needs account + ` twine upload ` . Remaining: PyPI upload itself (manual ), launch materials (HN/Twitter/Discord) . |
6668
6769### Files shipped (mapping tới plan §4 repo structure)
6870
0 commit comments