AutoScientists — Coordination-Mechanism Testbed Exemplar

A deterministic, ablatable reference implementation of the five coordination mechanisms from Gao, Fang & Zitnik, AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation (arXiv:2605.28655): a shared champion/experiment-log state, a dead-end registry, effect-size ranking, noise-band confirmation, and stagnation-driven team reorganization.

It is a microscope on the coordination primitives, not a reproduction of the full agentic system. The deterministic core is fully tested; a live HermesProposer (Ollama) drops into the same loop for an opt-in agentic demo.

When to use this template

Use this template when you need to isolate and measure agent-coordination mechanisms: coordinated teams vs single-thread baselines under matched budgets, deterministic ablation studies, and honest per-mechanism effect reporting (including nulls). Choose template_autoresearch_project instead if your focus is end-to-end AutoResearch loop infrastructure (evidence registries, review gates, artifact integrity) rather than coordination primitives. Full roster: projects/AGENTS.md.

Publication and rendering

The publishing metadata and per-platform status below are compiled from docs/manuscript/config.yaml by infrastructure.publishing.status_report — do not hand-edit between the markers; update the config and regenerate (see the legend).

A Deterministic Testbed for Self-Organizing Agent-Team Coordination · v1.0.2 · MIT · Daniel Ari Friedman

Concept DOI: 10.5281/zenodo.20533669 | Version DOI: 10.5281/zenodo.20931927 | Repository: docxology/template_autoscientists

Publishing surface — 20 platforms, 9 published:

Platform	Tier	Status	Reference	Credentials
zenodo	first-class	✅ published	10.5281/zenodo.20533669	ZENODO_API_TOKEN
github	first-class	✅ published	docxology/template_autoscientists	GITHUB_TOKEN
arxiv	first-class	⚪ available	—	—
pypi	first-class	✅ published	https://test.pypi.org/project/template-autoscientists/1.0.2/	PYPI_TOKEN, TESTPYPI_TOKEN
ipfs_pinata	first-class	✅ published	https://gateway.pinata.cloud/ipfs/QmX6F3ExeC87DxxH95YgJuHcJmjhXuf1x54BKF6V5bn5z4	PINATA_JWT
ipfs_web3storage	first-class	⚪ available	—	WEB3_STORAGE_TOKEN
software_heritage	first-class	✅ published	https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/docxology/template_autoscientists	—
github_pages	first-class	✅ published	https://docxology.github.io/template_autoscientists/	GITHUB_TOKEN
cloudflare_pages	first-class	⚪ available	—	CLOUDFLARE_API_TOKEN
netlify	first-class	✅ published	https://6a44401a17ba160047501e98--tranquil-kleicha-0c9203.netlify.app	NETLIFY_AUTH_TOKEN
huggingface_hub	first-class	✅ published	https://huggingface.co/datasets/ActiveInference/template_autoscientists	HUGGINGFACE_TOKEN, HF_TOKEN
osf	first-class	✅ published	https://osf.io/6xcf5/	OSF_TOKEN
amazon_kdp	documented	🟡 planned	—	AMAZON_KDP_EMAIL, AMAZON_KDP_PASSWORD
google_play_books	documented	🟡 planned	—	GOOGLE_PLAY_BOOKS_SERVICE_ACCOUNT_JSON
gumroad	documented	🟡 planned	—	GUMROAD_ACCESS_TOKEN
leanpub	documented	🟡 planned	—	LEANPUB_API_KEY
lulu	documented	🟡 planned	—	LULU_CLIENT_KEY, LULU_CLIENT_SECRET
draft2digital	documented	🟡 planned	—	DRAFT2DIGITAL_API_TOKEN
stripe	documented	🟡 planned	—	STRIPE_SECRET_KEY, STRIPE_PUBLISHABLE_KEY
ingramspark	documented	🟡 planned	—	INGRAMSPARK_CLIENT_ID, INGRAMSPARK_CLIENT_SECRET

Keywords: agent coordination, scientific discovery, noise-band confirmation, ablation study, reproducible research, language-model agents.

Status legend: ✅ published (durable identifier recorded in config.yaml) · 🔵 reserved (identifier reserved but not yet registered by final publication) · ⚪ available (adapter implemented and locally verifiable) · 🟡 planned. This block is generated — edit docs/manuscript/config.yaml, then regenerate with uv run python -m infrastructure.publishing.status_report --project <path> --write.

The 3 platforms still shown ⚪ available are not automatable to "published" with current tooling/credentials, not an oversight: arXiv has no submission API in this codebase (infrastructure.publishing.arxiv only prepares a local tarball — a human must upload it via arxiv.org and the resulting arxiv URL would then be added to publication.published_artifacts); Cloudflare Pages needs a CLOUDFLARE_ACCOUNT_ID the configured API token cannot auto-discover; IPFS (Web3.Storage) has no WEB3_STORAGE_TOKEN configured.

• Canonical renderer: docxology/template with --project templates/template_autoscientists
• Tracked outputs: output/ in this project and output/templates/template_autoscientists/ in the monorepo; public output files above 50 MB stay out of git.

To regenerate this exemplar from the public monorepo:

git clone https://github.com/docxology/template
cd template
uv sync
./run.sh --project templates/template_autoscientists --pipeline --core-only
uv run python scripts/pipeline/stage_04_validate.py --project templates/template_autoscientists
uv run python scripts/pipeline/stage_05_copy.py --project templates/template_autoscientists

Standalone repositories are publication mirrors for source, DOI metadata, and tracked rendered artifacts. Use the monorepo above when you need the full shared infrastructure, pipeline stages, or cross-template validation.

What it honestly shows

The testbed is constructed so that no performance claim would be honest, and it makes none. Under a matched sequential experiment budget, coordinated teams partition one budget rather than adding parallel compute, so they cannot beat a single thread on the final answer. The measured results (regenerated by the scripts below) separate three honest axes:

Axis	Mechanism	Measured effect on this objective
Final answer	team partitioning	None. Every configuration reaches the clean optimum (clean metric 0.0); coordinated−baseline advantage is exactly 0.0. Partitioning across 3 teams even reaches the optimum a few experiments later (exp 16 vs 12) — coordination is not faster here.
Search hygiene	dead-end registry	Real. The registry-consulting proposer cuts redundant re-probes of retired directions from 36 → 0 and halts at 36 experiments instead of burning all 60 — same clean answer, less waste.
Noise robustness	noise-band confirmation	Real. ~13× smaller reported-vs-clean gap (0.0156 → 0.0012).
—	effect-size ranking, reorganization	Null on this objective. Identical to full coordination; reported plainly.

These magnitudes are properties of this synthetic objective, budget, and deterministic proposer — they illustrate the measurement, not a universal constant.

Quick Start

# Coordinated vs single-thread baseline (matched budget) + summary JSON/figure
uv run python scripts/run_search_comparison.py

# Per-mechanism ablation (metric figure + efficiency/hygiene figure)
uv run python scripts/run_ablation.py

# Deterministic test suite (live Hermes path is opt-in, see below)
uv run pytest tests -m "not requires_ollama"

Prerequisites & verification

Test/coverage gate (authoritative per-project command). Exit code 0 alone is not proof — confirm tests collected > 0 and coverage clears the 90% src/ gate (read the TOTAL line of the report for the live figure):

uv run pytest tests \
  -m "not requires_ollama" \
  --cov=src --cov-fail-under=90

Full pipeline render (qualified name required). This exemplar lives under projects/templates/, so the pipeline must be invoked with the qualified project name — a bare name fails environment setup:

uv run python scripts/runner/execute_pipeline.py --project templates/template_autoscientists --core-only

The language-model plug-in seam

The coordination loop depends only on the Proposer protocol (src/agents.py). Two real implementations are provided:

• DeterministicProposer — rule-based, fully deterministic, drives the tested exemplar runs and rendered figures. It consults the dead-end registry (the avoid set) so retired directions actually steer the search.
• HermesProposer — the live agent. It renders the shared state (champion, recent experiments, retired directions) to a prompt and asks a Hermes model served by Ollama which axis/step to try next, then parses the structured reply. Swapping it in is the only change needed to turn the deterministic reference run into a live agentic one.

Run the live path (requires ollama serve + ollama pull hermes3):

uv run pytest tests/test_hermes_live.py -m requires_ollama

The live network boundary is # pragma: no cover and excluded from the gate — the deterministic core is the gated surface.

No-mocks policy

Every test uses real deterministic objects and computations — no MagicMock, no mocker.patch. The "alternating proposer" and Hermes reply-parser tests are genuine objects exercising real code paths.

More information

See AGENTS.md for module-level technical documentation and the manuscript under docs/manuscript/ for the full write-up.

Template integrity

• Forward backlog: TODO.md.
• Copy-and-customize config: docs/manuscript/config.yaml.example.
• Project validation: uv run pytest tests/ --cov=src --cov-fail-under=90.
• Repo drift validation: uv run python scripts/audit/check_template_drift.py --strict.