Repository navigation
fix(core): free every pooled, dummy and scratch buffer on executor dispose - #61
Conversation
| private free = new Map<string, GpuBufferRef[]>(); | ||
| /** Every buffer this pool made, free or in use, so dispose() frees the ones | ||
| * a captured frame or a caller still holds. */ | ||
| private made: GpuBufferRef[] = []; |
There was a problem hiding this comment.
This stack branches from before #58, which added trim() to the pool, so it needs a rebase from #60 up (add/add conflicts in bufferPool.test.ts and eval.gpu.test.ts, content conflicts in bufferPool.ts, eval.ts, replayCache.ts).
One thing to fix during the rebase: made keeps every buffer trim() destroys, so it grows by one set per input length forever. Could the pool track only the buffers in use instead (an inUse set: add on acquire, delete on release), and have dispose() destroy free plus inUse? Then trim() needs no change and each buffer sits in exactly one place. Plus a test: trim then dispose destroys each buffer once.
| dispose: () => pool.dispose(), | ||
| // Forgets every handle, dummy and scratch it destroys, so a later | ||
| // dispatch makes fresh ones instead of binding a destroyed buffer. | ||
| dispose: () => { |
There was a problem hiding this comment.
A captured executor stays frozen after dispose, so it can neither replay nor eval. Reset frozen here too, or note that a captured executor is done after dispose?
3b5f012 to
5cda021
Compare
1b64fc3 to
fabb69c
Compare
… the executor fixes leave
… made, so trim() leaves nothing behind
… its old replays throw
fabb69c to
bd6f02b
Compare
Description
Closes the dispose leaks #60 found in the executor.
BufferPooltracks the buffers it has handed out and not taken back, sodispose()also frees ones still in use: a captured frame's target or atoArrayresult nobody released. Each buffer is either free or in use, never both, sotrim()(from fix(core): trim the buffer pool so varying input lengths stop piling up GPU memory #58) stays as it is and nothing it destroys is kept or destroyed twice. New check:trim then dispose destroys each buffer once.gpuExecutor.dispose()frees its f16/f32 dummies and every scratch buffer, including replaced scratches memoized handles may still bind.dispose()also forgets the handles, dummies and scratches it destroys, so an executor used again after it (asbench/ab.tsdoes after every case) makes fresh ones instead of binding a destroyed buffer. It also ends a capture: the executor can eval and capture again, and areplay()recorded beforedispose()throws instead of binding destroyed buffers. New checks:runs again after disposeandruns again after a captured frame is disposed.The matching checks in
eval.gpu.test.ts,replayCache.gpu.test.tsandzoo/leaks.gpu.test.tsdrop those buffers from their expected leftovers.Still left (next PRs)
gatherRowsuploads its index list per dispatch and never destroys it.toArrayon an eager graph never returns the result buffer to the pool.MinilmModel.forward) or its selector (embedPooledBatch). Its ids and positions are thegatherRowsleak above.cachedUniform: fixed in fix(core): take every kernel uniform from cachedUniform so none outlives its executor #62.replayCache.dispose()disposing each bucket's executor is not in this PR any more: #58 made the same fix onmain.Introduces a breaking change?
Type of change
Testing instructions
pnpm --filter runntime exec playwright install chromium(once per machine)pnpm --filter runntime testChecklist