While exploring trace evaluation for Agentsy’s manually interpreted workflow, I found a mismatch in the public standard contract grader. This is a static source observation at fc2810eea716726b0d3ac10f47a0877f9154cb34, not a reproduced mounted-runtime failure or the cause of application-side handoff mistakes.
The grader accepts a completed run, but its errors require forme.manifest.json and treat absent bindings/ as no outputs. Its extraction instructions likewise read only bindings.
The documented mounted layout instead records compiled intent, published world-model outputs and receipts. Standalone functions still return through bindings. Following the grader literally could therefore reject a documented mounted run before evaluating any clauses.
Please make artifact discovery distinguish these run kinds, resolve their declared outputs and preserve per-clause grading. Report missing or ambiguous evidence explicitly; do not choose whichever directory happens to yield a passing grade.
Proposed regression cases:
- A completed mounted fixture contains source snapshots, compiled intent, published output/version, matching receipts and a valid completion log, but no legacy manifest or bindings. The grader finds the output and evaluates its clauses.
- A standalone-function fixture returns through bindings. The grader evaluates its return contract without imposing mounted-state requirements.
- Remove a required published output or make its version inconsistent with the mounted receipt. Report the evidence gap; a completion log alone must not produce a satisfied verdict.
- An unsupported or mixed layout yields an explicit diagnostic.
These fixtures are proposed, not reported as executed. The change belongs in the standard contract grader and focused regression coverage; it needs no new VM semantics.
Related: #179 covers the inspector’s analogous discovery problem. This report concerns the separate contract-grader instructions; coordinate their layout handling. Fresh all-state issue searches for contract-grader and "contract grader" found no exact issue, and the matching PR search returned only the older package migration.
While exploring trace evaluation for Agentsy’s manually interpreted workflow, I found a mismatch in the public standard contract grader. This is a static source observation at
fc2810eea716726b0d3ac10f47a0877f9154cb34, not a reproduced mounted-runtime failure or the cause of application-side handoff mistakes.The grader accepts a completed run, but its errors require
forme.manifest.jsonand treat absentbindings/as no outputs. Its extraction instructions likewise read only bindings.The documented mounted layout instead records compiled intent, published world-model outputs and receipts. Standalone functions still return through bindings. Following the grader literally could therefore reject a documented mounted run before evaluating any clauses.
Please make artifact discovery distinguish these run kinds, resolve their declared outputs and preserve per-clause grading. Report missing or ambiguous evidence explicitly; do not choose whichever directory happens to yield a passing grade.
Proposed regression cases:
These fixtures are proposed, not reported as executed. The change belongs in the standard contract grader and focused regression coverage; it needs no new VM semantics.
Related: #179 covers the inspector’s analogous discovery problem. This report concerns the separate contract-grader instructions; coordinate their layout handling. Fresh all-state issue searches for
contract-graderand"contract grader"found no exact issue, and the matching PR search returned only the older package migration.