Local-First YouTube Retrieval-Augmented Generation (RAG) System Project Overview This repository contains a monolithic RAG application designed to enable contextual conversations with YouTube video content without relying on external cloud APIs. The system architecture addresses the dual challenges of data privacy and the high costs associated with proprietary LLM tokens by keeping all processing local to the user's hardware.
Core Engineering Decisions
-
Hybrid Transcription Pipeline To ensure the system could run effectively on consumer-grade hardware (specifically optimized for high-performance mobile CPUs), I integrated the open-source implementation of OpenAI’s Whisper. By selecting the base and tiny model variants, I established a sustainable balance between transcription accuracy and thermal management, mitigating the "FP16" latency common on non-GPU systems.
-
Local Vector Space Management The system utilizes FAISS (Facebook AI Similarity Search) for local vector indexing instead of a managed cloud database. This decision minimizes external dependencies and allows for sub-second similarity searches within the transcript metadata, keeping the entire monolith self-contained and performant.
-
Handling System-Level Dependencies The project demonstrates the successful integration of Python-based machine learning logic with system-level binaries (FFmpeg). The environment is configured to handle real-time audio extraction and stream processing, ensuring a seamless data pipeline from the raw YouTube URL to the embedding engine.
Technical Stack Language: Python 3.11
Orchestration: LangChain
Models: OpenAI Whisper (Transcription), Hugging Face (Sentence Embeddings)
Vector Store: FAISS
Interface: Streamlit
Deployment and Testing During testing, the application successfully processed complex technical lectures, accurately retrieving context and mapping user queries to specific video segments. The system proved resilient across multi-minute transcription cycles, maintaining stability and efficient memory allocation throughout the RAG lifecycle.
Implementation Footnote This project was developed as a case study in building privacy-first AI tools. It focuses on the practical application of large-scale open-source models within the constraints of local hardware environments.