Repository navigation
feat: add outcome learning and repository-aware memory retrieval - #71
Conversation
Code Review SummaryStatus: No Issues Found | Recommendation: Merge The incremental commit on top of Correctness / Safety FindingsNo correctness or safety findings. The new validation in The new test The CHANGELOG and Ponytail ReviewThe diff is the minimum required to add the new retryable semantic. No new abstractions, no duplicated helpers, no speculative config, no tests of mocks/framework behavior. The duplicated Ponytail: Lean already. Ship. Ponytail net: 0 lines. Suggested Minimal PatchNo patch needed. Final Merge GuidanceCan merge as-is. The lexical-catchup retryability change is correctly scoped, the validation guards are defensive-only, and the new test exercises the real recall + rebuild flow end-to-end. Files Reviewed (4 files)
Fix these issues in Kilo Cloud Previous Review Summaries (4 snapshots, latest commit cd7eb8b)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit cd7eb8b)Status: No Issues Found | Recommendation: Merge Overview
VerdictApprove — the Previously Flagged Findings — Verification on HEAD
|
| Severity | Count |
|---|---|
| CRITICAL | 4 |
| WARNING | 7 |
| SUGGESTION | 0 |
Verdict
Request changes — incremental commit 4462fd6 adds SQLite DATABASE_IN_USE retry translation and benchmark ergonomics metrics but does not touch the previously flagged migration-33 backfill / replay-UNIQUE defects. Every prior critical finding is still present on HEAD.
Issue Details (click to expand)
CRITICAL (still present on HEAD 4462fd6)
| File | Line | Issue |
|---|---|---|
daem0nmcp/event_store.py |
937 | payload["record"].get("worked") KeyError on every legacy memory.created row in migration 33 backfill. |
daem0nmcp/event_store.py |
862 | _project_provenance raises ValueError("memory provenance is invalid") on any non-conforming legacy event; aborts the entire backfill. |
daem0nmcp/migrations/schema.py |
2655 | FK on memory_outcome_signals.record_id aborts backfill on orphan stream_id. |
daem0nmcp/verification_v7.py |
53 | import_event_bundle re-inserts provenance edges/signals, hitting UNIQUE constraint and breaking verify-v7. |
WARNING (still present on HEAD 4462fd6)
| File | Line | Issue |
|---|---|---|
daem0nmcp/api/v7/federated_retrieval.py |
470 | explore intent skips retention even when retention_mode=apply; RETENTION_APPLIED is misleading. |
daem0nmcp/retrieval/late_interaction.py |
106 | Non-standard ColBERT normalization (max-abs + L2); breaks MaxSim ranking vs reference. |
daem0nmcp/retrieval/runtime.py |
487 | Missing fallback to embedding reranker when late-interaction profile is not ready; silent rerank loss. |
daem0nmcp/retrieval/composer.py |
170 | query_identifiers produces py instead of foo.py; code-binding match broken for path queries. |
daem0nmcp/config.py |
120 | retrieval_utility_mode default shadow adds query overhead by default; mismatch with dataclass default. |
daem0nmcp/migrations/schema.py |
3264 | DDL + backfill run inside one BEGIN; long write lock stalls readers. |
daem0nmcp/api/v7/gateway_operations.py |
158 | Three model_validate/dict accesses on the post-adapter result sit OUTSIDE the try/except blocks. |
Files Reviewed (incremental, 7 files)
daem0nmcp/api/v7/discovery_operations.py(+27 / −9) —_translate_errorbusy/locked mapping,_lexical_catchup_pending_syncand_generation_is_current_syncnow returnDATABASE_IN_USEfor retryable contention. Verified previous CRITICAL/WARNING findings in other modules still apply.benchmarks/autoresearch_benchmark.py(+152) —_ergonomics_metricsboots a per-mode temp dir + v7 server, asserts the listed-tool / write-call / destructive-challenge call counts are stable. Hardcoded contract values are intentional.tests/api_v7/test_discovery_operations.py(+432 / −145) — six new retryable/non-retryable SQLite contention tests, refactored_entity_retrieval_resulthelper. Logic matches_translate_errormapping.tests/api_v7/test_gateway_operations.py(+146 / −56) — destructive-challenge retry test extended tocoreandfulltool surfaces, plus a newValueErroradapter-exception case. Matchesbind_gateway_handlerssanitisation path.docs/v7-operations.md,CHANGELOG.md,README.md— doc refinements describing the same behaviours reviewed above.
No new correctness or safety issues were identified in the incremental diff. The 11 previously raised findings are unchanged on HEAD.
Ponytail Review
The incremental commit is mostly additive wiring (retry-translation + tests). No new abstractions or one-implementation interfaces introduced. One minor robustness note (already covered by existing test coverage, not raised as a new finding):
daem0nmcp/api/v7/discovery_operations.py:713-716shrink (optional): the busy/locked substring checkany(word in str(error).casefold() for word in ("busy", "locked"))is fragile — sqlite3 OperationalError messages unrelated to lock contention that happen to contain those substrings would be misclassified. Standardise on the actual prefix (str(error).startswith(("database is locked", "database is busy"))) or check the sqlite3sqlite3_errstrcodes 5/6.benchmarks/autoresearch_benchmark.py:54-71shrink (optional): the strict-mode probe discards the firstserverand rebuilds it; the rebuild has no test-visible side effect beyond theprobeClient exiting. Inline the probe call into the post-rebuild flow so a single server instance is used.
Ponytail: already lean in the productive directions. The two notes above are optional cleanups, not blockers.
Ponytail net: 0 lines.
Suggested Minimal Patch
No new patches required from the incremental commit. The four CRITICAL and seven WARNING patches already documented in the previous summary remain the smallest safe change set before merge:
event_store.py— guardpayload["record"].get("worked")with a typed lookup; loosen_project_provenanceto skip non-conforming legacy payloads with a logged warning; switch the two new INSERTs toINSERT OR IGNORE.migrations/schema.py:2655— drop thememory_outcome_signals.record_idFK or backfill in dependency order.verification_v7.py:53/import_event_bundle— make provenance projections idempotent or skip them when the edges already exist.migrations/schema.py:3264— split DDL and backfill into separate transactions and stream the backfill withLIMIT.gateway_operations.py:158— move the post-adapter dict accesses andmodel_validatecalls inside the existingtry.federated_retrieval.py:470— route theexploreintent through_compose_retained.late_interaction.py:106— drop max-abs pre-normalization;runtime.py:487— add the embedding-reranker fallback;composer.py:170— add the basename form toquery_identifiers;config.py:120— alignretrieval_utility_modedefault.
Final Merge Guidance
Do not merge until the 4 CRITICAL findings (migration 33 backfill + replay UNIQUE conflicts) are fixed. The 7 WARNING-level items are correctness regressions and should also be addressed before merge.
Fix these issues in Kilo Cloud
Previous review (commit d0464b0)
Status: 11 Issues Found | Recommendation: Address before merge
Overview
| Severity | Count |
|---|---|
| CRITICAL | 4 |
| WARNING | 7 |
| SUGGESTION | 0 |
Verdict
Request changes — incremental commit d0464b0 adds CovenantCeremony and daem0n_tool_call gateway but inherits every critical migration-33 backfill defect and one new gateway hardening gap. The previous critical findings are still present on HEAD.
Issue Details (click to expand)
CRITICAL (carried from prior review; still present on HEAD d0464b0)
| File | Line | Issue |
|---|---|---|
daem0nmcp/event_store.py |
937 | payload["record"].get("worked") KeyError on every legacy memory.created row in migration 33 backfill. |
daem0nmcp/event_store.py |
862 | _project_provenance raises ValueError on non-conforming legacy events; aborts migration 33 backfill. |
daem0nmcp/migrations/schema.py |
2655 | FK on memory_outcome_signals.record_id aborts backfill on orphan stream_id. |
daem0nmcp/verification_v7.py |
53 | import_event_bundle re-inserts provenance edges/signals, hitting UNIQUE constraint and breaking verify-v7. |
WARNING (carried from prior review; still present on HEAD d0464b0)
| File | Line | Issue |
|---|---|---|
daem0nmcp/api/v7/federated_retrieval.py |
470 | explore intent skips retention even when retention_mode=apply; RETENTION_APPLIED is misleading. |
daem0nmcp/retrieval/late_interaction.py |
106 | Non-standard ColBERT normalization (max-abs + L2); breaks MaxSim ranking vs reference. |
daem0nmcp/retrieval/runtime.py |
487 | Missing fallback to embedding reranker when late-interaction profile is not ready; silent rerank loss. |
daem0nmcp/retrieval/composer.py |
170 | query_identifiers produces py instead of foo.py; code-binding match broken for path queries. |
daem0nmcp/config.py |
120 | retrieval_utility_mode default shadow adds query overhead by default; mismatch with dataclass default. |
daem0nmcp/migrations/schema.py |
3264 | DDL + backfill run inside one BEGIN; long write lock stalls readers. |
WARNING (new finding introduced by commit d0461310b...)
| File | Line | Issue |
|---|---|---|
daem0nmcp/api/v7/gateway_operations.py |
158 | ResponseMeta.model_validate(result["meta"]), ToolCallData(... data=result["data"]), and ApiError.model_validate(result["error"]) run OUTSIDE the try/except (ValidationError, TypeError) / except Exception blocks. Malformed adapter results bypass the sanitized gateway error envelope. |
Files Reviewed (incremental, 49 files)
daem0nmcp/api/v7/ceremony.py(new, 287 lines) —CovenantCeremony.admit; minor bare-except concerns only.daem0nmcp/api/v7/gateway_operations.py(new, 203 lines) —bind_gateway_handlers; 1 WARNING (post-adapter validation outside try).daem0nmcp/api/v7/{application,composition,fastmcp,middleware,models,pinned,policy,production,resource_repository,resources,responses,rule_trigger_operations,runtime_services,tools}.py— ceremony wiring, gateway binding, listed-tools middleware, type moves; no new high-severity findings.daem0nmcp/claude_hooks/{pre_edit,session_start,stop}.py— gateway reminder paths; reviewed.daem0nmcp/config.py—tool_surface/covenant_modedefaults.tests/api_v7/{test_ceremony,test_gateway_operations,test_models,test_responses,test_foreground_configuration}.py(new) — match thin contracts; no findings.
Ponytail Review
The incremental commit is mostly additive wiring; little redundant code. One shrink opportunity exists in the ceremony module:
daem0nmcp/api/v7/ceremony.py:212-213,247-248shrink: the two bareexcept Exception: return CeremonyOutcome(violation, notice, None)blocks hide every guidance/brief failure with no log. Either add a structured log line (logger.warning("ceremony guidance failed", exc_info=True)) or tighten to(RuntimeServiceError, ValidationError, OSError, TypeError, ValueError). -3 lines net.daem0nmcp/api/v7/fastmcp.py:583-658shrink:manifest.toolsis iterated five times to compute a subset check, an adapter map, and gateway registrations. Cachetool_names = frozenset(s.name for spec in manifest.tools)once after construction. -2 lines net.daem0nmcp/api/v7/policy.py:147delete:_PREFLIGHT_PLACEHOLDERremoval is safe but undocumented. Either delete the_EXCLUDEDcomment-only block above it, or leave as-is.
Optional:
daem0nmcp/api/v7/middleware.py:143-146shrink:ListedToolsMiddleware.on_list_toolsshould defensively handle non-iterablecall_nextreturns. Optional, future-proofing only.daem0nmcp/api/v7/production.py:506-507,579-580shrink:_active_databaseonly raisesProductionConfigurationError; drop the broadexcept Exceptionfallback in both_compact_brief_readerand_briefing_reader. Optional.
Ponytail net: ~-7 lines.
Suggested Minimal Patch
- Migration backfill (CRITICAL): in
daem0nmcp/event_store.py:_project_provenance, replace unconditionalpayload["record"].get("worked")with a typed lookup; loosen_project_provenanceto skip non-conforming legacy payloads with a logged warning rather than raising. Switch the two new INSERTs toINSERT OR IGNOREso re-runs are idempotent. - FK orphan (CRITICAL): in
daem0nmcp/migrations/schema.py:2655, dropmemory_outcome_signals.record_idFK or backfill in dependency order sostream_idalways exists inmemory_records. - Replay UNIQUE (CRITICAL): in
daem0nmcp/verification_v7.py:53, markmemory_provenance_edges/memory_outcome_signalsas out of the canonical-replay digest, or skip_project_provenancefor replayed events that already produced the edges. - Backfill transaction (WARNING): in
daem0nmcp/migrations/schema.py:3264, split DDL and backfill into separate transactions and batch the backfill withLIMIT. - Gateway hardening (new WARNING): move
result["meta"]/result["data"]/result["error"]accesses and the threemodel_validatecalls inside the existingtryso adapter malformations produce the sanitized gateway envelope. - Retention/intent (WARNING): route
exploreintent through_compose_retainedindaem0nmcp/api/v7/federated_retrieval.py:470. - Late interaction (WARNING): drop the max-abs pre-normalization in
daem0nmcp/retrieval/late_interaction.py:106; add the embedding reranker fallback indaem0nmcp/retrieval/runtime.py:487; fixquery_identifiersindaem0nmcp/retrieval/composer.py:170; alignretrieval_utility_modedefault indaem0nmcp/config.py:120.
Final Merge Guidance
Do not merge until the 4 CRITICAL findings (migration 33 backfill + replay UNIQUE conflicts) are fixed. The 7 WARNING-level items are correctness regressions and should also be addressed before merge; the new gateway_operations.py:158 finding is the smallest safe addition.
Fix these issues in Kilo Cloud
Previous review (commit cb74022)
Status: 10 Issues Found | Recommendation: Address before merge
Overview
| Severity | Count |
|---|---|
| CRITICAL | 4 |
| WARNING | 6 |
| SUGGESTION | 0 |
Verdict
Request changes — migration 33 backfill will crash on the very first legacy row, breaking the upgrade path. Plus a replay-breaking UNIQUE conflict.
Issue Details (click to expand)
CRITICAL
| File | Line | Issue |
|---|---|---|
daem0nmcp/event_store.py |
937 | payload["record"].get("worked") KeyError on every legacy memory.created row in migration 33 backfill. |
daem0nmcp/event_store.py |
862 | _project_provenance raises ValueError on non-conforming legacy events; aborts migration 33 backfill. |
daem0nmcp/migrations/schema.py |
2655 | FK on memory_outcome_signals.record_id aborts backfill on orphan stream_id. |
daem0nmcp/verification_v7.py |
53 | import_event_bundle re-inserts provenance edges/signals, hitting UNIQUE constraint and breaking verify-v7. |
WARNING
| File | Line | Issue |
|---|---|---|
daem0nmcp/api/v7/federated_retrieval.py |
470 | explore intent skips retention even when retention_mode=apply; RETENTION_APPLIED is misleading. |
daem0nmcp/retrieval/late_interaction.py |
106 | Non-standard ColBERT normalization (max-abs + L2); breaks MaxSim ranking vs reference. |
daem0nmcp/retrieval/runtime.py |
487 | Missing fallback to embedding reranker when late-interaction profile is not ready; silent rerank loss. |
daem0nmcp/retrieval/composer.py |
170 | query_identifiers produces py instead of foo.py; code-binding match broken for path queries. |
daem0nmcp/config.py |
118 | retrieval_utility_mode default shadow adds query overhead by default; mismatch with dataclass default. |
daem0nmcp/migrations/schema.py |
3264 | DDL + backfill run inside one BEGIN; long write lock stalls readers. |
Files Reviewed (53 files)
daem0nmcp/event_store.py- 2 criticaldaem0nmcp/migrations/schema.py- 1 critical, 1 warningdaem0nmcp/verification_v7.py- 1 criticaldaem0nmcp/api/v7/federated_retrieval.py- 1 warningdaem0nmcp/retrieval/late_interaction.py- 1 warningdaem0nmcp/retrieval/runtime.py- 1 warningdaem0nmcp/retrieval/composer.py- 1 warningdaem0nmcp/config.py- 1 warningbenchmarks/coding_memory_eval.py,daem0nmcp/api/v7/{discovery_operations,models,pinned,resource_repository,runtime_services,tools}.py,daem0nmcp/{capabilities,code_bindings,database,migrations/v7,models,retrieval/{dense_projection,repository,service,types,utility},schema_version,storage_activation}.py,docs/release/v7/{inventory.md,requirements.json},pyproject.toml,uv.lock,.gitignore,CHANGELOG.md,README.md, all listedtests/*.py— reviewed; no new high-severity findings.
Ponytail Review
Ponytail findings (selected; full list omitted for brevity):
daem0nmcp/event_store.py:806shrink: collapse 55-line_is_source_provenanceinto two short helpers.daem0nmcp/code_bindings.py:78reuse: replace hand-rolled_bindings_from_contextvalidation with a single Pydantic model.daem0nmcp/code_bindings.py:113yagni: dropbinding_refs_from_contextwrapper; only two callers.daem0nmcp/migrations/schema.py:3264shrink: drop manualEventCommand(...)reconstruction in_backfill_memory_learning_evidence; expose a_project_provenance_from_row.benchmarks/coding_memory_eval.py:419stdlib: replace hand-rolled_percentilewithstatistics.quantiles.benchmarks/coding_memory_eval.py:597shrink: drop redundantif topics < 1check.daem0nmcp/retrieval/service.py:697stdlib: usedelta.total_seconds() * 1_000_000instead of hand-rolled microseconds math.daem0nmcp/retrieval/composer.py:162shrink: simplifyquery_identifierswith a single regex.daem0nmcp/retrieval/late_interaction.py:79yagni: droplist(...)materialization of single-result model calls.daem0nmcp/api/v7/runtime_services.py:524reuse: inline_validate_informed_by; producer already runs the samelen(set(...))check.
Ponytail net: ~-90 lines.
Suggested Minimal Patch
- In
event_store.py, changepayload["record"].get("worked")torecord = payload.get("record") or {}+worked = record.get("worked") if isinstance(record, Mapping) else None. - In
migrations/schema.py:_backfill_memory_learning_evidence, wrapstore._project_provenance(...)intry/except ValueError(log + skip) and split DDL/backfill into separate transactions with batchedLIMIT. - In
event_store.py:_project_provenance, switch the two newINSERTs toINSERT OR IGNORE. - In
verification_v7.py/import_event_bundle, skip_project_provenancewhen the provenance edges/signals already exist for the(workspace_id, event_id). - Fix
_normalized_tokens(drop max-abs
[Snapshot truncated.]
Additional previous summary content was truncated to keep this comment within platform limits.
Reviewed by minimax-m3 · Input: 65.2K · Output: 16.3K · Cached: 2.1M
Review guidance: REVIEW.md from base branch working_branch
…osition smoke failed before/passed after under 256-token budget, unrelated paths stay unprioritized; 38 tests/28 subtests and scoped Ruff pass; frozen nDCG unchanged, no quality-gain claim
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…; same-size timestamp-preserving edit was falsely current before and needs_revalidation after. File and symbol regression covers edit/restoration, 16 focused tests and scoped Ruff pass. Frozen ranking metrics unchanged.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ty; full-path queries no longer promote other directories sharing the filename, or unrelated same-named symbols. Actual 256-token composition before retained wrong and right; after excludes wrong while retaining right for full/partial path and qualified symbol. 38 tests/38 subtests, scoped Ruff pass; canonical quality unchanged.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ated recall. Real before/after committed-source smoke preserved exact output digests and diagnostics; federated synchronous calls fell3->2 (one fewer baseline composition), non-explore shadow retained distinct variant. 50tests/53subtests and scoped Ruff pass. Primary metric unchanged; frozen workload has no explore-intent requests.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ource cache. Actual12contexts/1,050,000B readcalls12->1 (single-sample19.22ms->4.00ms, not guaranteed speedup); actualSQLite recall two distinct records shares one read and nextrecall detects restoredmtime edit withfresh read. Budget0/exactEOF/CRLF/partialrefusal/growthprobe/cacheeviction smokes pass.91affectedtests/79subtests plus13batchtests andscopedRuffpass. Qualityunchanged.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…le retainingexplicit./rootanchor. Actual256-token composition previouslylost ./ anddoubleslash targets andbackslash falselypromotedotherdir; nowaliases retainrightonly. ./router.ts remainsroot-only(notnestedrouter.ts), barefilename staysbroader.38tests/54subtests, scopedRuffpass; frozenqualityunchanged.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
… optionalinternalmetadata, preserve navigation/projectionidentity. Actualstaticmethod->classmethod changed runtimepayload->TypeError butbinding remainedcurrentbefore; afterneeds_revalidation. Producer nav/relationshipsidentical.112binding/learningtests,81index/discoverytests+1skip, scopedRuffgreen. Frozenqualityunchanged; olderdecoratedbindings conservativelyrevalidateonce.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…lds. Actualbefore: docs/[E99].md labeldenieddirectcontext; federationshippedASCII[E99] withmanifest[E1]only. After: directreturnscontext,label/contentfullwidth[E99], manifestonlyreal[E1]. Rawtypedpaths/canonicalrecords preserved, offsets/tokencountregressionspass.40tests57subtests, scopedRuffgreen; qualityunchanged.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…lqueryBM25unchanged/noindexrebuild. Actualheldoutprovider: meaningfulbody1 outranks16-tagdraft2, body6/rationale5 outrankmixed4; tag-only3 andties7/8preserved, allrawscoresbit-identical.54existing/newtests24subtests, Ruffgreen. FrozenprimarydidNOTimprove—earlier.85predictionnotobserved andnotclaimed.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ashseeds1/2/3 beforeproduceddifferentitem/manifestchannelarrays; afterallbyte-identicaldense,graph,lexical.24tests47subtests, scopedRuffgreen. Primarymetricsunchanged; canonicallexical-onlyseed0harnessdoesnotcovermultiproviderrestartdrift.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…tes to existing retryable CAPABILITY_DEGRADED, while unsupported extensions/missing symbols remain INVALID_ARGUMENT. Real absent-package store smoke changed INVALID_ARGUMENT to CAPABILITY_DEGRADED and production public envelope retryable=True. Atomic store/rebind regressions: 115 passed, 29 subtests; scoped Ruff clean after formatter. Frozen nDCG unchanged; kept for independently verified correctness, not ranking gain.
Result: {"status":"keep","coding_memory_ndcg":0.821533827904,"baseline_ndcg":0.671913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ion zero; tag-assisted tier and original diagnostic BM25 preserved. Real held-out both-body-match case reverses spam ahead of useful evidence while exact score mappings stay identical. Frozen harness nDCG .821533827904 -> .85, baseline .671913456093 -> .721913456093; all other reported metrics unchanged. 55tests/24subtests, scoped Ruff green; frozen fixtures/goldens untouched. Removed incidental strict retention-arm superiority assertion after better baseline also achieved fact retention1; no replacement or weakened golden.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…me-name definitions no longer silently aggregate. Real parser before ambiguous A.target/B.target capture succeeded, after rejects CodeBindingReferenceError; qualified sibling edit stays current. Existing alias becoming ambiguous is honestly unverifiable, same-identity spans preserved; store/rebind reject atomically. 124tests/30subtests plus scoped Ruff clean after formatting. Frozen nDCG .85 and all secondary metrics unchanged; kept for independent correctness.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…mental resolve/stat/read failures unverifiable (capture CAPABILITY_DEGRADED); definite missing/directory/escape remain changed. Actual normalized-identical LF3.6MB->CRLF5.4MB now unverifiable, notfalsechanged. Actual Windows exclusivefilelock now unverifiable/Unavailable capture, restoredsamefile current. Corrected Windows directory PermissionError distinction using observedmode; no extra nominalcapturestat. 146tests/30subtests,1platformskip; scopedRuffgreen afterformatter. Frozen metric .85 unchanged.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…ker parity as internal context. Actual production federated JSON+unbackedE99 acceptedbefore, rejectsValidationError after; real directSQLite recall still current->changed correctly acrosscalls. Fixed one inconsistent handcrafteddiscoveryfixture ratherthanaccommodatinguncitedtext. 78tests/95subtests, scopedRuffclean. Frozenmetric.85 allsecondariesunchanged; independentlyverified correctness.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
… during fit/resequence/export; requested-token baseline stays pre-policy, count/flags/prefixverification preserved. ActualCoreTokenizer256budget now retainsstalewarning+3currentitems194tokens vswarningomittedbefore;600budget same4items212vs476tokens. None/explore unchanged. 91tests/103subtests, scopedRuffclean afterstrictzip/formatter. Frozen .85 andallsecondarymetrics unchanged; independentpackingbenefit.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…e bytes avoids repeated memcmp and duplicatebyteownership, retainingresolve/stat/budget/freshread rules. Real4.5MB warmfile120evaluations/batch7samplemedian45.596->24.7523ms(localdescriptivemeasurement,notlatencyguarantee). Nextbatchedit needsrevalidation; realSQLite2recordrecall1readcurrent thennextcall1freshreadbothchanged.155tests/25subtests1skip, scopedRuffgreen. Canonical .85/secondariesunchanged; independentperformancebenefit.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
…dlineoffsets; root+namefingerprintkeys close rootlessqualificationcache. Actual12symbols strictparses12->1/read1, all12hashes EXACTequal; onlyeditedsymbolchanged. Actual__init__ nestedrootvalidrefsallcurrent; invalidouterqualificationininnerroot currentBEFORE ->needsrevalidationAFTER. Capturestill12parses/12reads intentionallyseparaterun20.183tests/55subtests1skip, scopedRuffgreen afterformatter. Frozen .85/secondarymetrics unchanged; independentcompute/correctnessbenefit.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
… preserveeveryrefvalidation/containment andfreshlatercalls. Realmutationafterfirstactualparse before mixedold/new ->after BOTHoriginalSHAs,filealreadyreplacement; real12symbolcapture reads/parses12->1 andallSHAs EXACTsame. Targeted154tests30subtests1skip; FULLintegrated3305passed66skip506warnings1627subtests. FullRuffcheck+format488files,mypy48APIfilesplatformLinux,releaseinventory77tools/10resources/586requirementsPASS. Finalfrozenprimary.85 vsbaseline.821533827904 +3.5%,allothermetricsexceptreferencebaselineunchanged. Cap20reached,no pushes.
Result: {"status":"keep","coding_memory_ndcg":0.85,"baseline_ndcg":0.721913456093,"outcome_reuse_ndcg":1,"provenance_chain_ndcg":1,"required_fact_retention":1,"stale_flag_recall":1,"false_flag_rate":0,"rendered_tokens_mean":95.733333333333,"listed_tool_count":9,"guided_write_calls":1,"destructive_challenge_calls":2,"strict_explicit_write_calls":3}
Sanitize malformed gateway output without revalidating caller input, distinguish SQLite contention by error code, and report explore retention as not applicable. Restore missing derived learning parents from validated full canonical replay before transactional schema 33 backfill; preserve strict authority validation and rollback. Add consumer regressions and clarify operator costs and retry remediation.
|
Addressed the current optional SQLite-classification finding in cd7eb8b: discovery reuses numeric SQLite BUSY/LOCKED classification, with only a narrow canonical-diagnostic fallback for code-less exceptions. A real missing-table error mentioning busy_records or locked_cache now remains nonretryable CAPABILITY_DEGRADED; real lock contention still yields retryable DATABASE_IN_USE. Consumer regressions passed. The benchmark-only server-rebuild suggestion is not adopted: the strict rejection probe and measured scenario intentionally use separate fresh server/session state, and the frozen harness remains unchanged. Re-running that exact harness after remediation preserved coding-memory nDCG 0.850000000000, required-fact retention/stale recall 1.0, false flags 0.0 and the 9 listed / 1 guided / 3 strict / 2 destructive-call measurements. All 11 inline review threads now have individual evidence-backed responses. Valid boundary/recovery/diagnostic findings are fixed; unsupported provenance skipping, conflict suppression, normalization rollback, silent reranker fallback and transaction splitting were not applied. Targeted checks passed (210 tests and 217 subtests), plus full Ruff/type/inventory gates. The final full suite and pushed-head Actions checks are still running; their final results will be added to the PR description. |
Bind entity-selection readiness to its observed lexical generation so first publication between retrieval and inspection cannot erase a transient miss. Keep same-generation indexed misses terminal, preserve SQLite classification, and cover the real retrieval/publication/identical-retry transition.
|
Follow-up CI correction pushed as b38753f. The Ubuntu/Python 3.10 failure occurred in initial entity-recall catchup, not the BEGIN EXCLUSIVE lock scenario. A controlled real Task8RecallService/lexical-builder smoke reproduced generation publication between an empty retrieval and readiness inspection: the old check returned terminal CAPABILITY_DEGRADED although an identical retry succeeded. The check now retains retryable DATABASE_IN_USE when publication advances the selected generation, while same-generation indexed genuine misses remain terminal. Original SQLite classification is unchanged. The real before/after smoke, discovery/process suite (44 tests, 20 subtests), full Ruff/format/mypy/inventory gates and unchanged frozen benchmark (nDCG 0.85) pass. Final-head full local suite and Actions matrix are running; no green-CI claim is made yet. |
|
Review remediation complete at b38753f (with cd7eb8b). All 11 inline threads are answered and resolved; the latest Kilo review reports no issues. All 12 Actions jobs now passed at this exact head, including every Ubuntu/Windows/macOS Python 3.10–3.12 test leg; GitGuardian and Kilo checks also passed. Runner-unassigned cancellations were rerun until the actual missing leg executed successfully. The valid gateway, malformed-history, derived-parent recovery, SQLite-classification and explore-diagnostic findings are fixed, and the CI-discovered lexical publication race is reproduced/fixed with a real consumer regression. Frozen benchmark remains coding-memory nDCG 0.85 with unchanged fact/stale/false-flag and call-count metrics. Explore composition remains baseline: only its diagnostic changed, not a routing through retained packing. Verification transparency: the optional-extras local final full runs each had 3,326 passes/66 skips/1,667 subtests and one failure (WinError 5 storage rename once; dreaming startup returned advertised retryable DATABASE_IN_USE in the instrumented run). Native sharing/SQLite/Qdrant/drain instrumentation across the real case, all 34 path-policy tests and the full-suite rename observed successful rename with no owned target handle. The original WinError owner is unknown after process exit; no speculative fix or retry/suppression was applied. These caveats are retained in the PR description rather than represented as clean local full-suite results. |
Summary
Migration and rollout
Review remediation (cd7eb8b, b38753f)
Supporting fixes
Verification
Frozen experiment evidence
Final coding-memory nDCG 0.850000000000, versus initial 0.821533827904 (~3.5% relative improvement). Reference-arm nDCG 0.721913456093 (initial 0.671913456093). Outcome reuse, provenance-chain quality, required-fact retention and stale-flag recall remain 1.0; false flags 0.0; mean rendered tokens 95.733333333333. The unchanged harness reproduced these metrics after review fixes. These are synthetic workload results, not broad latency or competitor claims.
Earlier synthetic evaluation evidence
24-topic lexical workload: outcome-reuse nDCG 0.650 → 1.000; provenance-chain nDCG 0.710 → 1.000 with trace credit; required procedure-fact retention 0% → 100%; stale-binding recall 100%, false flags 0%. Fully-enabled ColBERT overall nDCG was 0.840.
These are synthetic, not competitor comparisons. Eval utility weight is 0.2 (product default 0.1). Retention distractors preserve the prescribed text but use task tags and the same procedure category with empty steps, preventing category diversity from automatically rescuing the actionable procedure. All stress settings are exposed in report metadata.