Production-Oriented QLoRA Fine-Tuning & LLMOps Pipeline
Fine-tuned TinyLlama using QLoRA, PEFT, TRL, and the HuggingFace ecosystem
with a deployment-oriented inference architecture.

---

🚀 Overview

Agent-Forge is a production-oriented GenAI engineering project that implements the complete lifecycle of modern LLM adaptation and deployment — from raw dataset to containerized inference server.

🎯 Goal: Deeply understand and implement end-to-end LLM fine-tuning & deployment infrastructure.

🧠 ML Engineering QLoRA Fine-Tuning 4-bit NF4 Quantization PEFT / LoRA Adapters Supervised Fine-Tuning (SFT) Conversational Dataset Engineering

⚙️ LLMOps & Infra FastAPI Inference Serving Redis Caching Architecture Qdrant RAG Integration Docker Containerization Deployment-Ready Pipelines

---

🏗️ Architecture

  HuggingFace Dataset
          │
          ▼
  Conversational Formatting
          │
          ▼
  Tokenizer (TinyLlama)
          │
          ▼
  4-bit NF4 Quantization  ◄──── BitsAndBytes
          │
          ▼
  QLoRA + PEFT Adapter Injection
          │
          ▼
  SFT Training  (SFTTrainer / TRL)
          │
          ▼
  Inference Evaluation (Before vs After)
          │
          ▼
  LoRA Adapter Saving
          │
          ▼
  FastAPI Inference Server
          │
          ▼
  Redis + Qdrant + Docker Deployment

---

🧠 Base Model

Property	Value
🤖 Model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
📦 Parameters	1.1 Billion
⚡ Quantization	4-bit NF4
🎯 Fine-Tuning Method	QLoRA + PEFT

Why TinyLlama?

• ✅ Lightweight and experimentation-friendly
• ✅ QLoRA compatible out of the box
• ✅ Runs on free Colab GPU tiers
• ✅ Fast iteration cycles
• ✅ Ideal for mastering modern fine-tuning pipelines

---

⚡ QLoRA Configuration

Setting	Value
Quantization	4-bit NF4
Double Quantization	✅ Enabled
Compute Dtype	float16
Backend	BitsAndBytes

Benefits:

• 📉 Dramatically lower VRAM usage
• ⚡ Faster model loading
• 🧩 Memory-efficient training
• ✅ Colab-compatible fine-tuning

---

🧩 PEFT / LoRA Setup

Parameter	Value
Rank r	8
Alpha α	32
Dropout	0.05
Precision	float16
Quantization	4-bit NF4
Target Modules	q_proj, k_proj, v_proj, o_proj

---

📊 Training Metrics

╔══════════════════════════════════════════╗
║         Parameter Efficiency             ║
╠══════════════════════════════════════════╣
║  Trainable Params   :     2,252,800      ║
║  Total Params       : 1,102,301,184      ║
║  Trainable %        :        0.2044%     ║
╚══════════════════════════════════════════╝

💡 Key Insight: 99.8% of the base model remained completely frozen. Only lightweight LoRA adapters were trained — maximum efficiency, minimal compute.

---

📚 Dataset

bitext/Bitext-customer-support-llm-chatbot-training-dataset

Each training sample contains:

Field	Description
instruction	The user query / task
category	Domain category
intent	Intent classification
response	Target model response

Samples were transformed into instruction-tuned conversational format aligned with TinyLlama's chat template.

---

🔥 Training Pipeline

Dataset
  │
  ├─► Formatting  (conversational template)
  │
  ├─► Tokenization
  │
  ├─► QLoRA Adapter Injection
  │
  ├─► SFT Training  (via TRL SFTTrainer)
  │
  ├─► Inference Evaluation  (Before vs After)
  │
  └─► LoRA Adapter Saving

---

🧪 Evaluation Strategy

The same prompts were run in two passes to directly compare behavior:

 Base TinyLlama                 QLoRA Fine-Tuned TinyLlama
 ─────────────────              ────────────────────────────
 Generic responses       vs.    Domain-adapted responses
 No task grounding       vs.    Customer support grounded
 Prompt-agnostic         vs.    Instruction-tuned behavior

---

⚙️ Inference Server

Endpoint:

POST /generate
Content-Type: application/json

{
  "prompt": "How do I reset my password?"
}

Inference Flow:

Request → Tokenizer → TinyLlama + LoRA → Generation → JSON Response

---

🗂️ Redis Caching Layer

Planned caching layer for:

• ⚡ Repeated prompt acceleration
• 📉 Latency reduction
• 📈 Throughput optimization
• 💾 Inference result storage

---

🔎 Qdrant RAG Integration

Planned migration toward a Kubernetes Troubleshooting Assistant:

User Query
    │
    ▼
Embedding Model
    │
    ▼
Qdrant Vector Retrieval
    │
    ▼
Context Injection into Prompt
    │
    ▼
Fine-Tuned LLM → Response

---

🐳 Deployment Stack

Component	Technology
🌐 API Server	FastAPI
📦 Containerization	Docker
🗂️ Caching	Redis
🔍 Vector Store	Qdrant
🤗 Model Hosting	HuggingFace Spaces
🔄 CI/CD	GitHub Actions

---

📈 Key Learnings

🧠 ML Concepts

• Tokenization & chat templates
• 4-bit NF4 quantization mechanics
• QLoRA parameter efficiency
• PEFT & LoRA adapter training
• Transformer fine-tuning workflows
• SFTTrainer from TRL

⚙️ Engineering Concepts

• GPU memory optimization strategies
• HuggingFace ecosystem integration
• Dependency & version conflict debugging
• Inference evaluation pipelines
• Production LLMOps design patterns

---

🛣️ Roadmap

• 📡 vLLM Inference Serving
• 📊 Weights & Biases Experiment Tracking
• 🧪 MLflow Logging Integration
• 🔄 Streaming Responses via SSE
• 🧠 Multi-turn Conversation Memory
• ☸️ Kubernetes-Specialized Fine-Tuning
• 🚀 Fully Automated CI/CD Deployment

---

🛠️ Tech Stack

---

Made with 🔥 by Vivek Tyagi — AI Engineer

If you found this useful, drop a ⭐ — it means a lot!