BedtimeNews Agent

English | 中文

Agentic RAG (Retrieval-Augmented Generation) system for the 睡前消息 (BedtimeNews) knowledge base. Provides Q&A with automatic routing, semantic search, retrieved-transcript context, and episode citations.

Try it out: chat.bedtime.blog

Overview

This system indexes video transcripts from the BedtimeNews archive and enables semantic search with LLM-powered Q&A. Built with LangGraph, pluggable LLM/embedding providers (DeepSeek for chat and SiliconFlow's Qwen3 embeddings by default), and PostgreSQL + pgvector.

Key Features:

• Automatic query routing (archive retrieval vs constrained direct handling)
• Query optimization and semantic search
• LLM-based document grading
• Retrieved transcripts supplied as answer context, with markdown citations and citation repair
• Automated document indexing with incremental updates
• Web-based chat interface

Content Coverage

The system indexes video transcripts from bedtimenews-archive-contents covering diverse topics across multiple programs:

Program Catalog:

Catalog	Name	Description
main/	睡前消息	Comprehensive coverage across all topics
reference/	参考信息	Daily news aggregation
business/	产经破壁机	Economy, industry, business, technology
commercial/	讲点黑话	International relations, geopolitics
opinion/	高见	Technical analysis, infrastructure, engineering
daily/	每日新闻	Daily news updates
others/	其它文稿	Live Q&A and other related content

Topic Categories:

1. Domestic Economy & Industry - Economic policy, industrial development, real estate, local government debt, urban development
2. Technology & Innovation - AI, chips, semiconductors, autonomous vehicles, aerospace, engineering
3. Cross-border E-commerce & Global Expansion - SHEIN, TikTok, Chinese manufacturing advantages, global markets
4. Corporate Governance & Regulation - Corporate scandals, auditing, financial supervision, food safety, tax regulation
5. International Relations & Geopolitics - US-China relations, Russia-Ukraine conflict, Middle East, Korean Peninsula, Indo-Pacific
6. Social Issues & Civil Life - Education, healthcare, demographics, social welfare, urban governance
7. Cryptocurrency & Fintech - Bitcoin, blockchain, decentralized finance, digital assets
8. Population & Social Policy - Population crisis, socialized childcare, education system, social welfare reform
9. Infrastructure & Engineering - Railway construction, energy infrastructure, urban development, public utilities
10. Law & Judicial Affairs - Corporate disputes, criminal justice, consumer protection, regulatory frameworks

Architecture

Components:

• Caddy: Reverse proxy with automatic HTTPS (public deployments only)
• Frontend: Custom chat UI (static HTML/CSS/JS served by a small FastAPI app)
• Agent: LangGraph-based agentic RAG service
• Indexer: Automated document embedding pipeline
• Database: PostgreSQL with pgvector extension as vector database

For local testing, access the frontend directly at http://localhost:8000 (no Caddy).

Quick Start

Prerequisites

• Docker
• API keys for your chosen providers (by default: DEEPSEEK_API_KEY for chat and SILICONFLOW_API_KEY for embeddings)

Deployment Modes

This stack supports two deployment modes:

Local Testing (localhost)

Quick setup without public access or TLS:

# Start without Caddy (frontend on localhost:8000)
docker compose --profile local up -d

Access at http://localhost:8000. No domain, firewall, or TLS setup needed.

This runs the published images. If you have edited the code, add --build — see Published image vs. your checkout.

Public Deployment (recommended for production)

Public access with automatic HTTPS:

# Start with Caddy reverse proxy
docker compose --profile public up -d

Access at https://<your-domain>. Requires:

• DOMAIN in .env set to a domain you control (A record → this server's IP)
• ACME_EMAIL in .env for Let's Encrypt expiry notices
• Firewall open to the world on ports 80 and 443 (Let's Encrypt ACME challenges)

Important: This setup works for grey-cloud (DNS-only) at Cloudflare. If you enable orange-cloud proxy, see Cloudflare Setup below.

Setup

1. Clone the repository

   git clone https://github.com/zydo/bedtimenews-agent.git
   cd bedtimenews-agent

2. Configure environment

   Copy .env.example to .env and configure:

   cp .env.example .env
   # Edit .env 

   API keys are read from the shell environment, not from .env. .env holds non-secret config (provider/model selection, ports, DB settings); export your secrets in the shell instead, e.g.:

   export DEEPSEEK_API_KEY=...      # chat provider
   export SILICONFLOW_API_KEY=...   # embedding provider

3. Start services

   For local testing (no TLS):

   docker compose --profile local up -d

   For public deployment (with Caddy + TLS):

   docker compose --profile public up -d

4. Access the UI

   • Local: Open http://localhost:8000
   • Public: Open https://<your-domain> (e.g. https://chat.bedtime.blog)

   For public deployment: Caddy provisions/renews a Let's Encrypt certificate for DOMAIN. Requirements:

   • DOMAIN in .env — domain you control with A/AAAA record pointing at this server
   • ACME_EMAIL in .env — email for Let's Encrypt expiry notices
   • Firewall allows ports 80 and 443 from anywhere (inbound)

   Caddy handles HTTP→HTTPS redirects and automatic renewals.

Verify Installation

# Check service status
docker compose ps

# View logs
docker compose logs -f

Tests and Coverage

The root test command runs agent, indexer, and frontend in isolated processes:

uv run pytest
uv run pytest --cov

Options are forwarded to every component. To run only one component, invoke it from that directory:

cd agent  # or indexer / frontend
uv run pytest --cov

Releases

Tagged releases publish prebuilt multi-arch (amd64 + arm64) images to GHCR via release.yml:

• ghcr.io/zydo/bedtimenews-agent-agent
• ghcr.io/zydo/bedtimenews-agent-indexer
• ghcr.io/zydo/bedtimenews-agent-frontend

To deploy a published release, pin a version with IMAGE_TAG in .env (default latest) and pull:

IMAGE_TAG=0.1.0   # in .env, or leave as latest
docker compose --profile public pull
docker compose --profile public up -d

Published image vs. your checkout

docker compose up never builds on its own, even from a source checkout with local edits. The image: key decides what runs:

Situation	What docker compose up does
Tagged image already present locally	Reuses it — no pull, no build
Tagged image not present locally	Pulls the published image from GHCR
docker compose up --build	Builds from the checkout

So after editing code, rebuild explicitly or you will keep running the old image:

docker compose --profile local up -d --build agent web-local

Note that a locally built image and a published release share the same tag, so whichever was created last wins. docker compose pull overwrites a local build, and --build overwrites a pulled release.

To cut a release, push a v* tag (image tags drop the leading v):

git tag v0.1.0 && git push origin v0.1.0

Release notes should call out operational changes: new/renamed env vars, schema changes (e.g. EMBEDDING_DIM — see the runbook in indexer/README.md), and whether re-indexing is required. storage/postgres/init.sh only runs on a fresh data volume, so schema changes never apply automatically to existing deployments.

Cloudflare Setup

This configuration supports grey-cloud (DNS-only) at Cloudflare. The domain resolves directly to your origin server, and Let's Encrypt can reach it for ACME challenges.

If you enable orange-cloud proxy, the default Caddy setup will fail certificate renewal (Cloudflare terminates TLS, blocking ACME challenges). To use orange-cloud:

1. Grey-cloud initially: Obtain the Let's Encrypt cert first (as documented above).
2. Then switch to orange-cloud: HTTPS continues working for ~90 days using the existing cert.
3. Before cert expires (~90 days): Implement one of:
   • Cloudflare Origin Certificate: Generate a 15-year cert in Cloudflare dashboard, mount it into Caddy (recommended, simpler)
   • DNS-01 challenge: Add Cloudflare DNS plugin to Caddy for ACME via DNS API (keeps Let's Encrypt)

Cloudflare SSL/TLS mode: Set to Full (strict) when using orange-cloud with either approach above.

Service-Specific Documentation

• Frontend: UI customization
• Agent: API endpoints, Agentic RAG implementation
• Indexer: Document processing

Data Persistence

Data is persisted across restarts:

• PostgreSQL data (chunks + embeddings): bind-mounted to ./storage/postgres/volume
• Service logs: Docker named volumes bedtimenews_indexer_logs and bedtimenews_agent_logs

Project Structure

bedtimenews-agent/
├── agent/              # LangGraph agentic RAG service
│   ├── src/
│   ├── Dockerfile
│   └── README.md
├── frontend/           # Custom web UI (static + FastAPI)
│   ├── server.py       # FastAPI: serves static UI + proxies /chat SSE
│   ├── starters.py     # Sample questions data
│   ├── static/         # index.html, styles.css, app.js, logo
│   ├── Dockerfile
│   └── README.md
├── indexer/            # Document embedding pipeline
│   ├── src/
│   ├── Dockerfile
│   └── README.md
├── docs/diagrams/      # SVG architecture and workflow diagrams
├── storage/            # Database initialization scripts
│   └── postgres/
├── docker-compose.yml  # Service orchestration (with profiles)
├── Caddyfile           # Caddy reverse proxy configuration
├── .env                # Environment configuration (not in git)
├── .env.example        # Environment configuration template
├── THIRD_PARTY_NOTICES.md  # Third-party component licenses
└── README.md           # This file

License

MIT License — see LICENSE file.

This project uses Caddy (Apache-2.0 licensed) for automatic HTTPS in public deployments. See THIRD_PARTY_NOTICES.md for details.