Repository navigation
fix(nodejs): prevent cursor-file corruption and survive it at startup - #108
Closed
tangquanwei wants to merge 1 commit into
Closed
tangquanwei wants to merge 1 commit into
tangquanwei wants to merge 1 commit into
Conversation
A crash or overlapping write to the persisted cursor file could leave it zero-filled. FileStorage.get then rethrew the JSON.parse error from client.start(), which aborted the poll loop before it began — surfaced as "Poll error: Unexpected token ... is not valid JSON". - storage/file: atomic writes (temp + fsync + rename), serialize writes per key, tolerate a UTF-8 BOM, and quarantine corrupt files instead of throwing - transport/http: parse BOM/empty/non-JSON bodies safely; non-JSON responses become typed TransportErrors that are retried for 5xx/429 but not 4xx; network failures are now retryable - messaging: loadCursor/contextStore.load never throw; log cursor persist failures; skip redundant cursor writes - core/client: expose resetCursor() for recovery - pi-agent: auto-recover from a crashed poll loop (reset cursor + capped backoff) instead of going permanently silent - add tests for file storage, HTTP parsing, and poller cursor handling
Author
|
Superseded by a smaller, focused PR that fixes the same root cause: #109 (storage atomicity + corruption tolerance). Closing this to keep review scope tight; the transport/poller/pi-agent hardening can be split out separately if useful. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A single corrupt state file could permanently kill the bot.
Symptom
After this, the bridge stops receiving and replying, sets
connected = false, and never recovers.Root cause
~/.wechatbot/cursor.jsoncan end up zero-filled (or otherwise truncated).FileStorage.get()did a bareJSON.parse(raw)and rethrew on failure, soclient.start()rejected insidepoller.loadCursor()before the poll loop ever began. Three independent defects combined:FileStorage.set()used a plainwriteFile: non-atomic, nofsync, and the poller fires it without awaiting, so overlapping writes to the same path interleave.FileStorage.get()rethrew parse errors instead of tolerating a damaged file.bot.start().catch(...)and went permanently silent.Reproduction (old vs new)
Stress test: same path, 8 concurrent
set()calls, 3000 rounds, thenJSON.parsethe resulting file:A zero-filled file (e.g. 107 NUL bytes) specifically matches an unflushed write: the filesystem records the file size in metadata but the data is lost, so reads come back as zeros.
Changes
nodejs/src/storage/file.ts— stop producing corruption, and survive it anywayfsync→rename.delete/clearwait for in-flight writes.*.corrupt-<ts>) and treated as missing instead of throwing.nodejs/src/transport/http.ts,core/errors.ts— transport robustnessnull/ non-JSON bodies safely. Non-JSON responses become a typed,retryableTransportError(5xx/429 retried, 4xx not).nodejs/src/messaging/— never crash on bad stateloadCursor()andContextStore.load()never throw.nodejs/src/core/client.ts— exposeresetCursor()for recovery.pi-agent— auto-recover from a crashed poll loop (reset cursor + capped backoff) instead of going permanently silent.nodejs/package.json—npm testwasnode node_modules/.bin/vitest run, which fails on Windows (the.binentry is a shell script). Nowvitest run.Tests
tests/file-storage.test.ts— atomicity, concurrent writes, BOM, NUL/corrupt quarantine, delete-vs-write race.tests/http.test.ts— BOM/empty/null/non-JSON bodies, retry classification.tests/poller.test.ts— cursor load/persist resilience.77 → 102tests, all passing.tsc --noEmitclean.Notes
resetCursor()is additive.npm testchange is exactly what the CInpm teststep runs, so CI stays green.