examples: the cost of the calls on a failure, the peer's task held, scratch removed at exit - #24
Merged
Merged
Conversation
…er's task; scratch goes with the process examples/backends.py printed its table of calls only after the run returned, so a backend that failed after completed calls left the paid calls unreported. The table is printed on the way out of a failure too, when there were calls; a backend that fails before its first call still stops with the one line naming what is missing. notebooks/03_two_actors.ipynb started the researcher's run with `asyncio.create_task` and dropped the task. The task is held now and awaited where the peer's run is shown, so an error in the peer is raised on the page rather than logged at exit. examples/tools_and_gates.py and examples/notes_and_context.py make a scratch directory on import and never removed it. It is removed when the process ends; the notes example's two runs still share one directory, which is the point of it. ADR-0010 cited `examples/two_actors.py`, which is not in the tree; the two-actor example is the scripted notebook. Tests: the notebooks run, every code cell in order with top-level awaits; the scripted examples leave nothing behind; the live example's failure after a metered call prints that call.
…; the peer's error is raised on the page
…'s error A subtotal an attempt was missing from printed like a complete one, so a cut-off call read as free; the row now prints >= before it and the footer says so. A failed call's row carries its error instead of None / None. The two-actor notebook test asserts both cells it checks were met, so an added line cannot skip its checks.
Owner
Author
|
b829135: the call table prints |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups from the source review of the examples part (record
c22da7c9).examples/backends.pysays what the calls cost when a later one fails. The table of calls was printed only after the run returned, so a backend that failed after completed calls left the paid calls unreported. It is printed on the way out of a failure too, an interrupt included, when there were calls; a backend that fails before its first call still stops with the one line naming what is missing.notebooks/03_two_actors.ipynbholds the researcher's task. The escalation handler started the researcher's run withasyncio.create_taskand dropped the task, so a failure in the peer was the loop's to report at exit. The task is held and awaited where the peer's run is shown, so the error is raised on the page.tools_and_gates.pyandnotes_and_context.pymake one on import and never removed it; it is removed at exit, and the notes example's two runs still share it, which is the point of that example.examples/two_actors.py, which is not in the tree; the two-actor example is the scripted notebook, and nothing runs it against a live model.Tests: every notebook runs, code cell by code cell with top-level awaits, offline; a researcher whose backend raises settles the reviewer's ask naming the error and is then raised where the peer runs are shown; the scripted examples leave nothing behind under
TMPDIR; the live example's failure or interrupt after a metered call prints that call. Each arm fails on the code before this change.