Sandbox allowlist drifts from canvas-plugins/sandbox.py

[andhines]

Problem

The sandbox allowlist used by the plugin assistant is currently maintained by hand in two places:

• canvas-plugin-assistant/sandbox-allowlist.md — human-facing reference
• canvas-plugin-assistant/scripts/lint_sandbox.py — STDLIB_ALLOWED / THIRD_PARTY_ALLOWED dicts used by the pre-deploy linter

The actual source of truth is plugin_runner/sandbox.py in canvas-medical/canvas-plugins. The public docs page is also derived from that file and tends to lag.

Both of our local copies will silently drift from upstream — meaning the linter can either accept imports the runner rejects, or flag imports the runner now permits. There is currently no mechanism to detect or correct this drift.

Suggested fix

Add a scripts/sync_sandbox_allowlist.py that:

1. Fetches plugin_runner/sandbox.py from canvas-medical/canvas-plugins at a pinned ref (or main).
2. Parses the ALLOWED_MODULES / ALLOWED_GLOBALS (and related) structures out of its AST.
3. Regenerates both sandbox-allowlist.md and the allowlist dicts in lint_sandbox.py — or writes a shared JSON file both consume.

Run it in CI on a weekly cron (or on every push) and fail if the regenerated files differ from what's checked in. That surfaces drift as a PR rather than silently going stale.

Tradeoff

Parsing sandbox.py directly is slightly more brittle to upstream refactors of that one file than scraping the docs page would be, but it reflects actual runtime behavior — which is what we care about for the linter.

Context

Came up while reviewing #87 (better sandbox handling), which introduced the hand-maintained copies.

[rafael-abdala]

This drift has now happened in practice, so here's a measured diff to size it.

Versions: CPA 1.29.0 vs. plugin_runner/sandbox.py as shipped in canvas_cli 0.174.0. (0.198.7 is current, so the real gap is likely wider.)

How it surfaced: lint_sandbox.py failed a pre-deploy run on from django.db.transaction import atomic — but the runner allows it (sandbox.py THIRD_PARTY_MODULES["django.db.transaction"] = {"atomic", "on_commit", "on_rollback"}), and the plugin loads fine on a live instance (Successfully loaded plugin ... (15/15 handlers loaded)). So it's a pure false positive, and the only way to tell was reading the runner's source — which is the thing the linter exists to save you from.

Direction of the drift (the good news): I diffed both allowlists in full, and the linter is a strict subset of the runner — every discrepancy is a false positive, zero are false negatives. Nothing the linter permits is rejected by the runner. So today this is a usability/trust bug, not a safety hole. Worth asserting that direction in the proposed CI check, since the two failure modes deserve different severities.

Modules the runner allows that lint_sandbox.py omits entirely

Module	Names
django.db.transaction	atomic, on_commit, on_rollback
django.db.models.functions	Coalesce, CumeDist, DenseRank, FirstValue, Lag, LastValue, Lead, NthValue, Ntile, PercentRank, Rank, RowNumber, Trim
django.contrib.postgres.indexes	GinIndex
html	HTML, escape, unescape
traceback	format_exc
_strptime	(bare module)

Names omitted on modules that are already listed

Module	Missing names
django.db.models	BooleanField, CASCADE, DO_NOTHING, DateField, DateTimeField, DecimalField, F, FloatField, ForeignKey, Func, Index, JSONField, ManyToManyField, OneToOneField, RowRange, SET_NULL, TextField, UniqueConstraint, ValueRange, Window
dataclasses	is_dataclass, make_dataclass
datetime	time
json	JSONDecodeError
re	fullmatch
requests	codes
time	time_ns
typing	TYPE_CHECKING
urllib.parse	unquote

sandbox-allowlist.md has the same gaps (its django rows stop at django.db.models.query / django.utils.functional), which matches your point that both hand-maintained copies drift together.

One implementation note for the sync script

The two sources categorize modules differently, so a naive per-table diff reports phantom findings. dateutil, dateutil.relativedelta, and defusedxml.ElementTree live in the linter's THIRD_PARTY_ALLOWED but in the runner's STANDARD_LIBRARY_MODULES. Diffing STDLIB_ALLOWED against STANDARD_LIBRARY_MODULES table-for-table flags those three as both "missing" and "over-permissive" when they're neither. Flattening both sides to a single module -> names map before comparing makes the result clean — that's how the numbers above were produced.

Given the runner's structures are plain module-level dicts of str -> set[str], importing sandbox.py from the installed canvas_cli and reading STANDARD_LIBRARY_MODULES / THIRD_PARTY_MODULES directly is simpler than the AST parse suggested above, and pins the check to the CLI version the user will actually deploy with — which is arguably the more useful invariant than matching canvas-plugins@main.