Fix GPU memory waste in demo.py and a viewer scene-corruption bug - #104
Open
Zawaer wants to merge 3 commits into
Open
Fix GPU memory waste in demo.py and a viewer scene-corruption bug#104Zawaer wants to merge 3 commits into
Zawaer wants to merge 3 commits into
Conversation
- Keep the full image sequence on CPU (pinned) instead of eagerly copying it to GPU before inference. inference_streaming/ inference_windowed already slice off and transfer one frame at a time internally, so the eager copy just held thousands of frames' pixels on GPU for the ~1 frame ever in flight, wasting several GB of VRAM on long sequences for no benefit. - Load the checkpoint with map_location="cpu" instead of the target device, avoiding a transient double-memory-copy (checkpoint tensors on GPU + model params on CPU) during the state_dict load before the model is moved to device.
_regenerate_cameras() removed and recreated camera frustum handles when
the camera downsample slider changed, but never removed the paired
"camera_frame" axes FrameHandle that add_camera() also creates at
/frames/{step}/camera_frame. Re-adding a node at an already-live path
on repeated slider changes (or during save_video()'s per-frame replay)
eventually raises "Cannot assign to 'position'/'visible' on a removed
FrameHandle". Because viser replays its full history of scene edits to
every newly-connecting client, this error gets baked permanently into
that history -- once triggered, the viewer stays broken for all future
connections until the server process is restarted.
Track camera_frame handles the same way frustum handles already are,
and remove both before regenerating.
Also replaces the deprecated matplotlib.cm.get_cmap with
matplotlib.colormaps[...] (removed in newer matplotlib releases).
MSVC's default preprocessor and C++ standard aren't enough to compile these pybind11/torch extension bindings, failing the build outright on native Windows. Add the required MSVC flags (/std:c++20, /Zc:preprocessor), gated to sys.platform == 'win32' so gcc/clang builds on Linux/macOS are unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three independent fixes found while running
demo.pyend-to-end on a real capture (native Windows + WSL2):inference_streaming/inference_windowedalready slice off and transfer one frame at a time internally. On longer sequences this wastes several GB of VRAM holding pixels for frames that aren't being processed yet, and was directly responsible for OOMs on an 8GB GPU. Also switched the checkpoint load tomap_location="cpu"to avoid a transient double-memory-copy at load time._regenerate_cameras()(triggered by the camera downsample slider, and hit repeatedly duringsave_video()) removed and recreated frustum handles but never cleaned up the pairedcamera_frameaxesFrameHandleit also creates. Re-adding a node at an already-live path eventually raisesCannot assign to 'position'/'visible' on a removed FrameHandle. Since viser replays its full scene-edit history to every newly-connecting client, this error gets baked permanently into that history — once triggered, the viewer is broken for all future connections until the server process is restarted, with no way to recover the in-memory reconstruction short of a full rerun. Also fixed amatplotlib.cm.get_cmapdeprecation warning (removed in newer matplotlib)./std:c++20//Zc:preprocessorflags, gated tosys.platform == 'win32'so Linux/macOS builds are unaffected.Test plan
demo.pyin both--mode streamingand--mode windowedon native Windows through a full room-scan clip; confirmed GPU memory no longer includes the full image sequence and the run completes without the memory-growth OOM seen before.camera_framehandles.setup.pychange; confirmed thewin32-gated flags don't touch the Linux/macOS compile args path.