Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Publishes the site/ directory to GitHub Pages, minus its build tools. site/tools/build_data.py copies the offline-rendered
# frames into site/frames/ and the models' one-shot files, byte for byte, into site/live/; the page runs a live piece only
# inside <iframe sandbox="allow-scripts">. Pull requests run the data check only.
name: pages
on:
push:
branches: [main]
paths: ["site/**", ".github/workflows/pages.yml"]
pull_request:
paths: ["site/**", ".github/workflows/pages.yml"]
workflow_dispatch:
permissions:
contents: read
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Check the site data and renderers
run: node site/tools/check_site.mjs
deploy:
if: github.event_name != 'pull_request'
needs: check
runs-on: ubuntu-latest
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
environment:
name: github-pages
url: ${{ steps.deploy.outputs.page_url }}
steps:
- uses: actions/checkout@v4
- name: Stage the site without its tools
run: |
mkdir _site
cp -a site/. _site/
rm -rf _site/tools
- uses: actions/configure-pages@v5
- uses: actions/upload-pages-artifact@v3
with:
path: _site
- id: deploy
uses: actions/deploy-pages@v4
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ pointer, and new work lands here.
| Path | What |
| --- | --- |
| [`reports/`](reports/) | Written reports and method documents: test results, candidate and exclusion lists, and the bake-off dashboard (`reports/dashboard/`: a static HTML page and its `data.json`). |
| [`oneshots/`](oneshots/README.md) | Test 5 (brass orrery, SVG) and test 6 (voxel pagoda, three.js) one-shot replies: the model-written source, rendered frames and grades. **Untrusted model output**: read the README before opening any of it. |
| [`oneshots/`](oneshots/README.md) | Test 5 (brass orrery, SVG) and test 6 (voxel pagoda, three.js) one-shot replies: the model-written source, rendered frames and grades. Unedited model output; the site plays it in a sandboxed frame (see the README). |
| [`harness/`](harness/) | The harness: case runners (`run-case*.sh`, `run-matrix.sh`), graders and rubric (`grade-*.py`, `rubric.py`, `score-*.py`), the Home Assistant replay server and read-only proxy (`ha-replay.py`, `ha-readonly-proxy.py`, `ha-record.py`), the sandboxed eval box (`sandbox/`), test suites and prompts (`suites/`), entry and spec tooling (`tools/`, `engines/ENTRY-SPEC.md`, `engines/entry-specs/`), engine build and setup notes (`engines/`), and the harness's own tests (`tests/`). |
| [`site/`](site/data/README.md) | The project site: a data-driven page that renders `site/data/bakeoff.json` (built by `site/tools/build_data.py`) and plays the one-shots in a sandboxed frame. Published to GitHub Pages from `main`. |
| [`docs/model-research/`](docs/model-research/README.md) | Method, conclusions, per-model cards and the story test. Moved here from `agentic-toolbox`, which keeps only a pointer. |

## The tests in one paragraph each
Expand Down
23 changes: 11 additions & 12 deletions oneshots/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
# One-shot replies: orrery (test 5) and pagoda (test 6)

**Everything under `orrery/*/` and `pagoda/*/` except the frames is untrusted model output.**
Models wrote these SVG and HTML files in one reply, without tools, and nobody edited them. Treat
them like any file from the internet:

- Do not serve them from a web site, and do not turn this repository into a GitHub Pages site,
unless every model-written file sits behind a sandbox: an author-written viewer page that embeds
it in `<iframe sandbox="allow-scripts">`, and a Content-Security-Policy that sandboxes the file
itself and blocks fetch, XHR, WebSocket and loads from other hosts.
- `page.html` files load three.js from a public CDN (`meta.json` lists each page's external
URLs). Opening one in a normal browser makes that request and runs the model's script.
- To look at a result, use the frames (`*.webp`): they were rendered offline, in a container with
no network.
**Everything under `orrery/*/` and `pagoda/*/` except the frames is model output.** Models wrote these
SVG and HTML files in one reply, without tools, and nobody edited them. The operator read them and judged
the code safe to run, and the project site plays them (see below).

- The site (`site/`, published by `.github/workflows/pages.yml`) copies each `page.html` and `orrery.svg`
byte for byte into `site/live/` and runs it only inside `<iframe sandbox="allow-scripts">`, without
`allow-same-origin`, so a piece gets an opaque origin and cannot reach the site, its storage or the
top window. One piece runs at a time and only when a visitor opens it.
- `page.html` files load three.js from a public CDN, and a few load Google Fonts (`meta.json` lists each
page's external URLs). Opening one makes those requests and runs the model's script.
- The frames (`*.webp`) were rendered offline, in a container with no network. Grades come from them.

## Layout

Expand Down
Empty file added site/.nojekyll
Empty file.
Loading
Loading