fix: Recover from Sphinx parallel workers dying without a result

[hoxbro]

Description

Follow-up to #357, which did not work — the recovery path itself crashed:

AI Summary of what went wrong:

multiprocessing.BaseProcess.start() runs del self._target, self._args, self._kwargs to avoid a refcycle (bpo-30775), so proc._args is gone on exactly the processes the recovery targeted — every started worker. The build then died on an AttributeError instead of the original EOFError.

The fix takes a different approach, in a new nbsite/_parallel.py:

• ParallelTasks.__init__/add_task are patched to record the task function per task id at submit time. Sphinx keeps no reference to it anywhere, which is why fix: Improve EOF Error seen in HoloViews #357 tried to read it back off the dead Process.
• _join_one is reimplemented rather than wrapped, so the EOFError is handled at the pipe.recv() that raises it, where the failing task id is known. Wrapping the whole call loses that, and identifying the dead pipe afterwards means recv()-ing on live pipes and discarding real results.
• The dead worker's chunk is then re-run in the main process, mirroring ParallelTasks._process's exception handling so a genuine task failure still surfaces as SphinxParallelError rather than being swallowed.
• Normal bookkeeping (_result_funcs, _args, _precvs, _pworking) follows the untouched path, so waiting workers still get started and join() terminates. fix: Improve EOF Error seen in HoloViews #357 returned True without touching state, which could also spin.

Note this makes the build complete by retrying the chunk in-process. If the worker died from something deterministic in that chunk (e.g. a segfaulting C extension rather than an OOM kill) the main process will hit it too — but it will fail with a real traceback instead of a bare EOFError. The HoloViews symptom looks like resource pressure, so a retry should generally succeed.

Testing

nbsite/tests/test_parallel.py uses a task that calls os._exit(1) only when multiprocessing.parent_process() is not None, so it kills the worker but returns a distinguishable value when re-run in the main process. Covers: recovery of the dead worker's task, sibling tasks still completing, normal result collection, and a genuinely failing task still raising SphinxParallelError.

All 4 tests fail on main, and the pre-fix AttributeError above reproduces locally against main with the same scenario. Full suite: 52 passed.

AI Disclosure

Tool & Model: Claude Code + Opus 5
Usage: Diagnosed why #357 failed, wrote nbsite/_parallel.py and nbsite/tests/test_parallel.py.

• I have tested all AI-generated content in my PR.
• I take responsibility for all AI-generated content in my PR.

Checklist

• Tests added and are passing