encore_vm: clear registers when a call returns - #15
Merged
Merged
Conversation
The register file is a GC root set, and a call that returned (or failed) left whatever it last held in its registers. Those values stayed live until a later call happened to overwrite each register, so a host that calls the VM repeatedly carried the previous call's data through the next call's collections. In encore-benchmarks, W6 (COBS, 1024-byte frame) needed 53.5 KiB of heap for its timed run, although a run on a fresh VM fits in 40 KiB: the previous run's input, encoding and decoded lists were still reachable from stale registers. With the registers cleared, the case list fits in 32.25 KiB. The registers are reset after the top-level run of call_raw and call_address, on success and on error. They are left alone inside an extern, whose caller's registers are still in use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ALRGkKEshzfh8jxNFEeAmB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The 256-register file is scanned as GC roots, but a call that returned (or failed) left whatever it last held in its registers. Those values stayed live until a later call happened to overwrite each register, so a host calling the VM repeatedly carried the previous call's data through the next call's collections.
Found in encore-benchmarks: W6 (COBS, 1024-byte frame) failed its timed run at 40 KiB of heap even though the same run on the untimed pass fit. The previous run's input, encoding and decoded lists were still reachable from stale registers.
Fix
call_rawandcall_addressnow go throughrun_to_completion, which runsenter()and then resets the registers, on success and on error. The reset is skipped inside an extern, whose caller's registers are still in use.Tests
test_gc_ignores_registers_of_finished_calls: a 3-word heap that holds only one ctor; each call must reclaim the previous call's ctor, which only a stale register points to. It fails withHeapOverflowbefore the fix and passes after.cargo test --workspaceandcargo test --workspace --all-featurespass.cargo xtask minheap -w w6_cobs -v edrops from 53.5 KiB to 33,024 B.Once this is released (0.1.6?), encore-benchmarks will bump to it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ALRGkKEshzfh8jxNFEeAmB
Generated by Claude Code