🌿 autoNaturalist

Autonomous naturalist AI — an agent that participates in the GainForest Agent Arena (species identification + BioBlitz image review over real observation backlog) and a research track toward smarter ecological vision models, aligned with the INQUIRE benchmark (NeurIPS 2024).

Why

The GainForest Arena scores agents on real identification work: photo observations whose species label is missing or kingdom-rank. INQUIRE showed that CLIP-style models are far from expert-level on natural-world queries — especially behavior and context queries (CLIP ViT-H/14: 35.6 AP@50 fullrank; +GPT-4o reranking: 47.1). Species-rank ID from photos needs more than zero-shot embedding similarity.

Architecture

photo observation (ATProto PDS blob)
        │
        ▼
┌─────────────────────────────┐
│ Stage 1: Retrieval          │  BioCLIP / SigLIP embedding search
│ (candidate generation)      │  over species label space + iNat21 taxa
└─────────────────────────────┘
        │ top-K candidates
        ▼
┌─────────────────────────────┐
│ Stage 2: Rerank & Verify    │  LMM (vision) rerank with trait-level
│ (evidence-based ID)         │  justification — INQUIRE's biggest lever
└─────────────────────────────┘
        │ calibrated confidence + remarks
        ▼
┌─────────────────────────────┐
│ Arena Client                │  ATProto mutation API:
│                             │  app.gainforest.dwc.identification
│                             │  + tagged app.gainforest.feed.post reply
│                             │  (bare strongRefs, no nested $type!)
└─────────────────────────────┘

Lessons baked in (from arena run #1, agent ox-alpha)

• Vision is a prerequisite — metadata/vernacular-only derivation produced a wrong ID ("Jacaré Açu" → Melanosuchus niger; the photo was Caiman crocodilus).
• Bare strongRefs — nested $type inside reply.root/reply.parent breaks the indexer's thread filtering.
• Profile record first — new repos aren't indexed until app.certified.actor.profile exists (retroactive indexing after creation).
• Calibrated confidence only — Brier-style scoring punishes overconfident wrong IDs; skip below ~40% rather than spray.

Layout

src/autonaturalist/
  embedder.py       # BioCLIP/SigLIP candidate generation
  reranker.py       # LMM rerank w/ visible-trait verification (INQUIRE-Rerank style)
  identify.py       # end-to-end: photo -> species -> confidence -> remarks
  arena_client.py   # ATProto mutation API client (identification + tagged reply)
  bioblitz.py       # round math (7d rounds anchored 2026-07-04), flag submission
eval/
  inquire_rerank.py # harness against evendrow/INQUIRE-Rerank (HF)
docs/
  ARENA.md          # arena participation guide (skill.md mirror + gotchas)

Roadmap

1. Arena agent — vision-grounded ID pipeline, two-record submissions, heartbeat loop.
2. INQUIRE-Rerank eval — measure our reranker vs. leaderboard (GPT-4o = 62.4 overall).
3. Fine-tune on GainForest data — accepted steward IDs as supervision signal.
4. Behavior/context head — targeted improvements where CLIP fails hardest.

Setup

export GAINFOREST_API_KEY=<gf_pat_...>   # Settings → AI agent keys
pip install -r requirements.txt
python -m autonaturalist.identify --help

License

MIT