Skip to content

Add ai-gene-review case study for AI-assisted GO annotation review #82

Description

@dragon-ai-agent

Goal

Add documentation for ai4curation/ai-gene-review that fits the purpose of aidocs: practical guides for curators and knowledge-base maintainers who want to integrate AI into real curation workflows.

This should not read like an internal project README. It should present ai-gene-review as a concrete, reusable case study for AI-assisted Gene Ontology annotation review: what the workflow is, which harness components make it reliable, and what another KB team could adapt.

Audience fit

Primary readers:

  • GO, MOD, UniProt, and ontology curators evaluating AI-assisted annotation review
  • maintainers of GitHub-based knowledge bases who want a working YAML/LinkML curation pattern
  • developers supporting curator workflows with validation, citation checking, generated pages, and human review loops

The page should assume readers care about practical adoption: repository shape, commands, validation gates, evidence provenance, review policies, and how humans stay in control.

Suggested placement

A good structure would be:

  • add a tutorial or case-study page, for example docs/tutorials/ai-gene-review-go-annotation.md
  • add a short entry to docs/examples.md
  • cross-link from the existing YAML/LinkML pipeline guide where ai-gene-review is already mentioned

Capabilities to document

Cover the parts that map directly to aidocs themes:

  • Structured curation unit: one review per gene, stored as YAML under genes/<organism>/<gene>/<gene>-ai-review.yaml, validated against src/ai_gene_review/schema/gene_review.yaml.
  • Agent harness: checked-in instructions, domain-specific skills, reproducible just commands, validation-first workflow, and GitHub PR review rather than direct changes to production content.
  • Data acquisition: just fetch-gene <organism> <gene> / related commands for UniProt, GOA, publication caches, and review stubs.
  • Annotation review model: explicit actions such as ACCEPT, MODIFY, KEEP_AS_NON_CORE, MARK_AS_OVER_ANNOTATED, REMOVE, and UNDECIDED, with guidance on when not to overrule expert curators from incomplete evidence.
  • Evidence and provenance: cached publications, exact supporting text, notes files, reference review fields, and the distinction between literature evidence, GOA evidence, deep research, and local bioinformatics analyses.
  • Hallucination guardrails: ID+label term objects, LinkML term validation, reference validation, and the project-specific rule that machine-sourced GOA IDs are trusted while author-supplied core_functions term IDs are strictly validated.
  • Ontology-aware curation: OLS/OAK/QuickGO-style term lookup, avoidance of uninformative terms such as protein binding, and careful use of GO molecular function, biological process, and cellular component terms.
  • Computational prediction review: separate *-predictions-review.yaml workflow for model predictions that are not already in GOA, including categories such as correct novel, uncertain, paralog incorrect, and frequency-biased predictions.
  • Human feedback loop: rendered review pages, browser app, dashboards, voting/evaluation hooks, and use of community/expert feedback to improve reviews.

Suggested page outline

  1. What problem this solves: reviewing existing GO annotations with AI assistance while keeping curator judgment and validation central.
  2. Repository shape: show the gene folder layout and identify source-of-truth versus generated/cache files.
  3. A curator workflow: fetch gene data, read publications/deep research, update notes, review existing annotations, propose core functions, validate, render, open PR.
  4. Reliability mechanisms: term validation, reference validation, anti-hallucination ID+label pairs, validation commands, and PR/CI gates.
  5. What is reusable: schema-first YAML records, publication caches, explicit review-action enums, exact evidence snippets, task-specific skills, and generated review pages.
  6. What is domain-specific: GO guidelines, UniProt/GOA inputs, GO term branch constraints, and gene-function synthesis.
  7. Limitations and human checks: abstract-only publications, curator deference, over-annotation risk, and cases that should remain UNDECIDED.

Acceptance criteria

  • The documentation is written for curators/KB maintainers, not only developers of ai-gene-review.
  • The page explains how ai-gene-review demonstrates the broader aidocs patterns: agent harness, LinkML validation, evidence provenance, and human-in-the-loop review.
  • The page includes concrete commands and file paths from ai-gene-review.
  • The page links to the repo, schema, example gene review(s), validation commands, and rendered/browseable outputs where appropriate.
  • docs/examples.md includes ai-gene-review as an example repository.
  • mkdocs.yml navigation is updated if a new page is added.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions