AniFlow

Turn manhwa & webtoon chapters into narrated recap videos — fully automated.

_{The AniFlow web UI — pick a recap mode and run the pipeline end to end in your browser.}

_{Real output: frames from clips rendered by both pipelines — AI narration on clean panels, dialogue read with the original speech bubbles.}

---

AniFlow downloads manhwa/webtoon chapters, detects and crops panels using YOLO, extracts speech-bubble text, generates AI narration, converts it to speech, and renders video clips — all through a browser-based UI.

✨ Features

• 🌐 Web Scraper — download chapters from manhuaus, manhwatop, MangaDex, and more
• 🔍 YOLO Panel Detection — custom-trained models detect panels and speech bubbles
• 🧹 Text Removal — automatically clean speech bubbles from art panels
• 🤖 AI Narration — local (Ollama) or cloud (Gemini / Claude) narration
• 🎙️ Multi-TTS — Edge TTS (free), Kokoro (local), or ElevenLabs (cloud)
• 🎬 Video Rendering — Ken Burns animation, blur backgrounds, BGM, watermarks
• 👥 Character Casting — import from MyAnimeList or scan faces from chapters
• 📦 Batch Mode — process dozens of chapters unattended

Two Pipelines

	Dialogue Recap	Narrated Recap
Panels	Original panels with speech bubbles	Clean art panels (bubbles removed)
Audio	Extracted dialogue read aloud	AI-written YouTube-style recap narration
Best for	Letting the original dialogue shine	YouTube story recaps

📂 See it in action! Check the example_output/ folder for sample clips from both modes.

🎥 Video Tutorials

Tutorial	Link
💬 Dialogue Recap — Single Chapter	Watch on YouTube
💬 Dialogue Recap — Batch Mode (50+ Chapters)	Watch on YouTube
🎙️ Narrated Recap — Automatic Pipeline	Watch on YouTube
🎙️ Narrated Recap — Batch Mode (Full Series)	Watch on YouTube

---

📋 Requirements

Dependency	Purpose	Install
Python 3.10 – 3.12	Runtime	python.org
FFmpeg	Video/audio processing	brew install ffmpeg (macOS) · sudo apt install ffmpeg (Linux) · ffmpeg.org (Windows)
Ollama	Local AI narration & text extraction	ollama.com — or use 9router for narration only
~10 GB disk	Python env + model weights	—

GPU recommended but not required. Apple Silicon Macs and NVIDIA GPUs will significantly speed up panel detection and Kokoro TTS.

---

🚀 Quick Start

Option A: One-Command Setup (Recommended)

git clone https://github.com/aashish254/Aniflow.git
cd AniFlow
chmod +x setup.sh
./setup.sh

The setup script will check dependencies, create a virtual environment, install packages, pull Ollama models, and start the app.

Option B: Manual Setup

# 1. Clone the repository
git clone https://github.com/aashish254/Aniflow.git
cd AniFlow

# 2. Create a virtual environment (Python 3.12 recommended)
py -3.12 -m venv venv             # Windows
# python3 -m venv venv            # macOS / Linux

# 3. Activate virtual environment
# Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# macOS / Linux:
# source venv/bin/activate

# 4. Install Python dependencies (~10-20 min, torch is large)
pip install --upgrade pip
pip install -r requirements.txt

# 5. Configure 9router (Alternative to Ollama)
# If you use 9router, set these environment variables before running:
# Windows (PowerShell):
# $env:USE_NINE_ROUTER="1"
# $env:NINE_ROUTER_HOST="http://127.0.0.1:20128"
# $env:VISION_MODEL="Free-Tier"
# $env:WRITER_MODEL="Free-Tier"
# $env:TEXT_WRITER_MODEL="Free-Tier"

# 6. Pull Ollama models (Skip if using 9router)
# ollama pull qwen2.5vl:3b
# ollama pull qwen2.5:14b

# 7. Run the app
python app.py

# 8. Open UI in browser:
# http://localhost:8080

---

Windows Setup

# 1. Clone and enter the project
git clone https://github.com/aashish254/Aniflow.git
cd AniFlow

# 2. Run the setup script
setup.bat

Or follow the manual steps above using venv\Scripts\activate instead.

9router (alternative to Ollama)

9router exposes models through an OpenAI-compatible local endpoint, so you can run the narrator without pulling multi-gigabyte Ollama models. It replaces Ollama for AI narration only — speech-bubble text extraction still needs Ollama.

Start 9router first (default http://127.0.0.1:20128), then point AniFlow at it:

# Windows (PowerShell)
.\venv\Scripts\Activate.ps1

$env:USE_NINE_ROUTER = "1"
$env:NINE_ROUTER_HOST = "http://127.0.0.1:20128"
$env:VISION_MODEL = "Free-Tier"
$env:WRITER_MODEL = "Free-Tier"
$env:TEXT_WRITER_MODEL = "Free-Tier"

# Only if your 9router instance requires one:
# $env:NINE_ROUTER_API_KEY = "your_key"

python app.py

# macOS / Linux
source venv/bin/activate

export USE_NINE_ROUTER=1
export NINE_ROUTER_HOST=http://127.0.0.1:20128
export VISION_MODEL=Free-Tier
export WRITER_MODEL=Free-Tier
export TEXT_WRITER_MODEL=Free-Tier

python app.py

The same variables can live in .env instead — uncomment the 9router block in .env.example. Once USE_NINE_ROUTER=1 is set, the app's model health check queries 9router's /v1/models endpoint instead of Ollama's /api/tags.

---

🎬 How to Use

Narrated Recap (Recommended)

1. Source — Paste a manhwa URL → Analyze → pick chapters → Download (or point to a local folder of images)
2. Cast (optional) — Import characters from MyAnimeList, or scan chapter faces and assign names. The narrator uses real names ("Jin-Woo…") instead of "the dark-haired guy."
3. Stitch — Stitch → Detect (bubble panels) → Crop → Stitch → Detect (art panels) → Crop. Boxes are editable — drag, resize, or draw new ones.
4. Review — Check cropped panels; delete junk or re-crop.
5. Extract — AI reads speech-bubble text.
6. Narrate — Choose your AI backend:
   • 🦙 Ollama (local) — free & offline. qwen2.5vl:7b (vision) or qwen2.5:14b (text-only).
   • ✨ Gemini API — paste multiple API keys (auto-rotation on quota). Free keys at aistudio.google.com.
   • 🤖 Claude API — single key.
7. Review Text — Edit any narration line or re-narrate individual panels.
8. Audio — TTS with Edge (free), Kokoro (local), or ElevenLabs.
9. Video — Renders clips with Ken Burns animation, blur backgrounds, BGM.
10. Review Clips / Sort Clips — Preview, download, or batch-merge.

Dialogue Recap

Same flow minus the cast/narrate steps — extracted dialogue becomes the audio, panels keep their original speech bubbles.

Batch Mode

Download chapters 1–68, enable Batch Mode, hit Run Full Batch — every step runs across all chapters automatically. Progress is streamed live.

---

📁 Project Structure

AniFlow/
├── app.py                  # Flask web app + API routes
├── config.py               # All configuration & path settings
├── requirements.txt        # Python dependencies
├── setup.sh / setup.bat    # One-command setup scripts
├── start.sh                # Quick-start script (macOS/Linux)
│
├── pipeline/               # Core processing modules
│   ├── web_scraper.py      #   Chapter downloading (multi-site)
│   ├── stitcher.py         #   Image stitching
│   ├── panel_extractor_yolo.py  # YOLO panel detection
│   ├── cropper.py          #   Panel cropping
│   ├── text_extractor.py   #   OCR / text extraction
│   ├── text_remover.py     #   Speech bubble cleaning
│   ├── narrator.py         #   Local AI narration (Ollama)
│   ├── gemini_narrator.py  #   Cloud narration (Gemini API)
│   ├── scene_narrator.py   #   Scene-level narration
│   ├── tts_engine.py       #   Edge TTS engine
│   ├── kokoro_tts.py       #   Kokoro local TTS
│   ├── elevenlabs_tts.py   #   ElevenLabs cloud TTS
│   ├── compositor.py       #   Video rendering
│   ├── orchestrator.py     #   Pipeline orchestration
│   ├── cast_system.py      #   Character management
│   ├── character_builder.py #  Face detection & grouping
│   ├── mal_character_importer.py # MAL character import
│   ├── script_rewriter.py  #   Narration style rewriter
│   ├── capcut_director.py  #   CapCut export
│   └── panel_filter.py     #   Panel quality filtering
│
├── models/                 # YOLO model weights (committed)
│   ├── manhwa_panel_cropped.pt      # Art panel detector
│   └── panel_with_bubble_text.pt    # Bubble panel detector
│
├── templates/              # HTML templates
├── static/                 # CSS, JS, logos
├── assets/                 # BGM, watermarks, intros, outros
└── tests/                  # Test & diagnostic scripts

Generated Folders (gitignored)

Folder	Contents
manhwa_download/	Downloaded chapter images + per-chapter state
manhwa_stitch/	Stitched full-page strips
manhwa_detect/	Detection metadata (bounding boxes)
manhwa_crop/	Cropped individual panels
manhwa_text/	Extracted dialogue JSON
manhwa_audio/	Generated TTS audio files
manhwa_clips/	Final rendered video clips
output/	Legacy output directory
~/RECAP/Cast Data/	Your character casts (faces + profiles)

All generated data is regenerable — delete any folder and re-run the pipeline.

---

⚙️ Configuration

API Keys (Optional)

Copy .env.example to .env and fill in keys only for services you want to use:

cp .env.example .env

Key	Service	Required?
GEMINI_API_KEY	Google Gemini narration	Only if using Gemini
ANTHROPIC_API_KEY	Claude narration	Only if using Claude
ELEVENLABS_API_KEY	ElevenLabs TTS	Only if using ElevenLabs
MAL_CLIENT_ID	MyAnimeList import	Only if importing characters from MAL
NINE_ROUTER_API_KEY	9router narration backend	Only if your 9router instance requires one

Ollama + Edge TTS work completely offline — no API keys needed for the core pipeline.

Ollama Models

Model	Role	Size	Required?
qwen2.5vl:3b	Vision parser (reads panels)	~2 GB	✅ Required
qwen2.5:14b	Text-only narrator	~9 GB	📌 Recommended
qwen2.5vl:7b	Vision narrator (sees panels + cast)	~5 GB	Optional

Override any of these with the VISION_MODEL, WRITER_MODEL and TEXT_WRITER_MODEL environment variables. With 9router you will want to — it addresses models by its own tier names (e.g. Free-Tier) rather than Ollama tags, and the health check only reports the model as available when the name matches one 9router returns.

---

🔧 Troubleshooting

Problem	Solution
"Address already in use"	Another instance is running: lsof -ti :8080 | xargs kill -9
Ollama dot is red	Start Ollama and pull models: ollama pull qwen2.5vl:3b
Downloads hang	Protected site — the app auto-switches to headless browser (~40s). MangaDex URLs are most reliable.
Bubbles not removed	EasyOCR inpainter handles clear bubbles; complex ones can be fixed with the crop tool in Review.
ModuleNotFoundError	Make sure your venv is activated: source venv/bin/activate
Torch install fails	Try: pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

---

🙏 Acknowledgments & Credits

AniFlow is built on the shoulders of many incredible open-source projects. Huge thanks to:

AI & Machine Learning

• Ollama — Local LLM runtime for AI narration and text extraction
• Qwen 2.5 VL by Alibaba Cloud — Vision-language model for panel understanding
• Ultralytics YOLOv8 — Object detection framework for panel detection
• Magi v2 by Ragav Sachdeva — Manga/comic page understanding and speaker-aware transcription
• EasyOCR by JaidedAI — Text detection for speech bubble cleaning
• Manga OCR by kha-white — OCR tuned for manga/manhwa text
• Google Gemini API — Cloud AI narration alternative
• Anthropic Claude API — Cloud AI narration alternative

Text-to-Speech

• Edge TTS by rany2 — Free Microsoft Edge TTS (default engine)
• Kokoro TTS by hexgrad — High-quality local text-to-speech
• ElevenLabs — Premium cloud TTS

Video & Image Processing

• MoviePy by Zulko — Python video editing
• FFmpeg — The backbone of all media processing
• OpenCV — Computer vision operations
• Pillow — Image manipulation

Web Scraping

• Beautiful Soup by Leonard Richardson — HTML parsing
• curl_cffi — Cloudflare-friendly HTTP with TLS fingerprinting
• CloudScraper by VeNoMouS — Cloudflare challenge bypass
• DrissionPage by g1879 — Headless browser fallback for protected sites

Web Framework

• Flask — Python web framework
• Flask-SocketIO — Real-time progress streaming

Other

• lbpcascade_animeface by nagadomi — Anime/manga face detection cascade
• PyTorch — Deep learning framework
• Hugging Face Transformers — Model loading for Magi v2
• MyAnimeList — Character data source for the cast system

---

📜 License

This project is licensed under the MIT License.

---

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

---

Made with ❤️ for the manhwa community