fix(web): show image tool results instead of empty success

[tt-a1i]

Problem

Fixes #471.

A tool result that contains only a Pi image content block was projected to an empty string. The Web transcript then showed that result as a successful call with “no output”, even though the model had received the image. Text mixed with images kept the text and dropped the pictures. Unsupported blocks and images past the display budget disappeared without a separate explanation.

Value

The operator can check the same visual evidence the model received: how many images, their media types, and whether the page is omitting bytes because preview is unavailable or because the image exceeds the 10 MB display limit. A failed tool stays failed. An actually empty result still says it has no output.

Approach

projectMessage already omitted image bytes. It now also keeps a bounded media type and name, marks payloads over the existing prompt-image budget as bytesOmitted, and counts retained blocks it cannot render as unsupportedParts. Snapshot and live tool events both use that projection.

The transcript summary for read and other tool results shows that evidence directly. The body lists each image and states that bytes stay in the session and are not placed in the page. No new file access and no base64 in the DOM or snapshot.

Validation

• tests/web/tool-result-images.spec.ts renders the real Transcript with the reported image-only read result. Before the UI change the visible summary was readReadshot.pngReturned and did not mention the image. It now requires the image count, media type, preview or display-limit state, success versus failure, and the absence of the image bytes. All 7 cases pass.
• tests/web/protocol.test.ts checks the same projection drops image bytes and counts an unsupported block.
• tests/web/tool-result-images.e2e.ts loads an isolated in-memory session through WebHost and checks the built UI after refresh. The image summary is visible, the payload is absent from the snapshot and the page, and a reload keeps the same evidence.
• tests/web/plan-card.spec.ts and tests/web/app-render.spec.ts were re-run after the follow-up that keeps a text-only unpaired result on its existing output row (138 tests, all passed).
• bun run check passed, including the committed Web build.
• Local bun run test on Node 22.14 stopped in tests/extensions/shared/setup-config.test.ts on an ExperimentalWarning about type stripping (node:7555). That warning is from this VM’s Node, which is older than CI’s 22.19.0, and the assertion is unrelated to image tool results. The node suite otherwise reported 2658 pass / 1 fail / 9 skip and did not reach Vitest. CI on 22.19.0, 24, and 26 is the full-suite check.

Impact

• User-visible behavior: image tool results no longer appear as success with no output. Preview stays unavailable; oversized images and tool failures are separate states.
• Model-visible context/tools: unchanged. Image bytes are still not copied onto the Web snapshot or SSE payload.
• Runtime/lifecycle: unchanged.
• Persisted config/data: unchanged. Session records still hold the original image bytes.
• Compatibility or risk: clients that ignore unknown message fields are unaffected. unsupportedParts and bytesOmitted are additive.

  

[cursor]

Manual check of the same real Web session on both revisions. No UI fixture and no injected transcript.

Each side ran node bin/openpi.js web from that checkout, with its own agent directory. pi list showed one OpenPI source: /tmp/openpi-main at 708a44b for the before side, and /workspace at 56dbd5a for the PR head. A loopback OpenAI-compatible provider (fake-provider / fake-reasoner) answered the first completion with a real Pi read of workspace file shot.png (64×48 PNG). The follow-up provider request on both hosts contained one image_url part, image/png, 190 payload characters. The /api/snapshot body on both hosts contained neither the PNG signature nor base64,.

The browser typed “Please read shot.png.” and the transcript below is that settled turn.

Before, 708a44b: the read is Returned and the visible result is only Read image file [image/png]. The page does not say that an image was received.

main 708a44b transcript after reading shot.png

After, 56dbd5a: the same read shows 1 image · image/png · preview unavailable, and the body says the model received the image while the bytes stay in the session.

PR 56dbd5a transcript after reading shot.png

No code change from this check.