A production-ready template for building AI agent backends with FastAPI and LangGraph. Handles the hard parts — stateful conversations, long-term memory, tool calling, observability, rate limiting, auth — so you can focus on your agent logic.
Built for AI engineers who want a solid foundation, not a tutorial project.
- LangGraph stateful agent with checkpointing, tool calling, and human-in-the-loop support
- Long-term memory via mem0 + pgvector — semantic search per user, cache-backed
- LLM service with circular model fallback, exponential backoff retries, and total timeout budget
- Langfuse tracing on all LLM calls; Prometheus metrics + Grafana dashboards
- JWT auth with session management; rate limiting via slowapi
- Alembic migrations; optional Valkey/Redis cache layer
- Structured logging with request/session/user context on every line
git clone <repo-url> my-agent && cd my-agent
cp .env.example .env.development # fill in your keys
make install
make docker-up # starts API + PostgreSQLOpen http://localhost:8000/docs to see the interactive API.
For local development without Docker see docs/getting-started.md.
| Guide | What it covers |
|---|---|
| Getting Started | Prerequisites, local setup, first API call |
| Architecture | System design, request flow, component diagrams |
| Configuration | All environment variables with defaults |
| Authentication | JWT flow, sessions, endpoint reference |
| Database & Migrations | Schema, Alembic migrations, pgvector |
| LLM Service | Models, retries, fallback, timeout budget |
| Memory | mem0 long-term memory, cache layer |
| Observability | Langfuse, structured logging, Prometheus, profiling |
| Evaluation | Eval framework, custom metrics, reports |
| Docker | Docker, Compose, full monitoring stack |
app/
api/v1/ # Route handlers
core/
langgraph/ # Agent graph + tools
prompts/ # System prompt template
cache.py # Valkey/Redis + in-memory fallback
config.py # Settings
middleware.py # Metrics, logging context, profiling
limiter.py # Rate limiting
models/ # SQLModel ORM models
schemas/ # Pydantic request/response schemas
services/ # LLM, database, memory services
alembic/ # Database migrations
evals/ # LLM evaluation framework
PRs welcome. Please read docs/getting-started.md to get your environment set up, then follow the coding conventions in AGENTS.md.
Report security issues privately — see SECURITY.md.
See LICENSE.