diff --git a/01_getting_started/04_dependencies/.env.example b/01_getting_started/04_dependencies/.env.example new file mode 100644 index 0000000..6089699 --- /dev/null +++ b/01_getting_started/04_dependencies/.env.example @@ -0,0 +1,3 @@ +# RUNPOD_API_KEY=your_api_key_here +# PORT=8888 +# LOG_LEVEL=INFO diff --git a/01_getting_started/04_dependencies/.flashignore b/01_getting_started/04_dependencies/.flashignore new file mode 100644 index 0000000..ea5988c --- /dev/null +++ b/01_getting_started/04_dependencies/.flashignore @@ -0,0 +1,40 @@ +# Flash Build Ignore Patterns + +# Python cache +__pycache__/ +*.pyc + +# Virtual environments +venv/ +.venv/ +env/ + +# IDE +.vscode/ +.idea/ + +# Environment files +.env +.env.local + +# Git +.git/ +.gitignore + +# Build artifacts +dist/ +build/ +*.egg-info/ + +# Flash resources +.tetra_resources.pkl + +# Tests +tests/ +test_*.py +*_test.py + +# Documentation +docs/ +*.md +!README.md diff --git a/01_getting_started/04_dependencies/.gitignore b/01_getting_started/04_dependencies/.gitignore new file mode 100644 index 0000000..9e84778 --- /dev/null +++ b/01_getting_started/04_dependencies/.gitignore @@ -0,0 +1,44 @@ +# Python +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +env/ +venv/ +.venv/ +ENV/ +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +*.egg-info/ +.installed.cfg +*.egg + +# IDEs +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# Environment +.env +.env.local + +# Flash +.tetra_resources.pkl +dist/ + +# OS +.DS_Store +Thumbs.db diff --git a/01_getting_started/04_dependencies/README.md b/01_getting_started/04_dependencies/README.md new file mode 100644 index 0000000..ff46230 --- /dev/null +++ b/01_getting_started/04_dependencies/README.md @@ -0,0 +1,654 @@ +# 04 - Dependency Management + +Learn how to manage Python packages and system dependencies in Flash workers. + +## What This Demonstrates + +- **Python dependencies** - Installing packages with version constraints +- **System dependencies** - Installing apt packages (ffmpeg, libgl1, etc.) +- **Version pinning** - Reproducible builds with exact versions +- **Dependency optimization** - Minimizing cold start time +- **Input validation** - Using Pydantic field validators for data quality + +## Quick Start + +**Prerequisites**: Complete the [repository setup](../../README.md#quick-start) first (clone, `make dev`, set API key). + +### Run This Example + +```bash +cd 01_getting_started/04_dependencies +flash run +``` + +Server starts at http://localhost:8888 + +### Alternative: Standalone Setup + +If you haven't run the repository-wide setup: + +```bash +# Install dependencies +pip install -r requirements.txt + +# Set API key (choose one): +export RUNPOD_API_KEY=your_api_key_here +# OR create .env file: +echo "RUNPOD_API_KEY=your_api_key_here" > .env + +# Run +flash run +``` + +## Dependency Types + +### 1. Python Dependencies + +Specified in `@remote` decorator: + +```python +@remote( + resource_config=config, + dependencies=[ + "torch==2.1.0", # Exact version + "Pillow>=10.0.0", # Minimum version + "numpy<2.0.0", # Maximum version + "requests", # Latest version + ] +) +async def my_function(data: dict) -> dict: + import torch + import PIL + # Your code here +``` + +### 2. System Dependencies + +Install apt packages: + +```python +@remote( + resource_config=config, + dependencies=["opencv-python"], + system_dependencies=["ffmpeg", "libgl1", "graphviz"] +) +async def process_video(data: dict) -> dict: + import cv2 + import subprocess + + # FFmpeg available + subprocess.run(["ffmpeg", "-version"]) + + # OpenCV works (needs libgl1) + cap = cv2.VideoCapture("video.mp4") +``` + +### 3. No Dependencies + +Fastest cold start: + +```python +@remote(resource_config=config) # No dependencies! +async def simple_function(data: dict) -> dict: + # Only Python stdlib + import json + import re + from datetime import datetime + return {"result": "processed"} +``` + +## Input Validation with Pydantic + +Flash uses FastAPI and Pydantic for request validation. Validate inputs at the API layer before they reach your worker functions. + +### Why Validate? + +- **Prevent errors** - Catch invalid data before processing +- **Better error messages** - Clear feedback to API consumers +- **Type safety** - Enforce data structure and types +- **Documentation** - Pydantic models auto-generate API docs + +### Basic Validation + +Define request models with type hints: + +```python +from pydantic import BaseModel + +class DataRequest(BaseModel): + """Request model with automatic validation.""" + data: list[list[int]] # List of lists of integers + threshold: float = 0.5 # Optional with default +``` + +FastAPI automatically: +- Validates types (returns 422 if invalid) +- Generates OpenAPI docs +- Provides helpful error messages + +### Field Validators + +Use `@field_validator` for custom validation logic: + +```python +from pydantic import BaseModel, field_validator + +class DataRequest(BaseModel): + data: list[list[int]] + + @field_validator("data") + @classmethod + def validate_two_columns(cls, v): + if not v: + raise ValueError("Data cannot be empty") + + # Require at least 2 rows for statistics + if len(v) < 2: + raise ValueError( + f"Need at least 2 rows to compute statistics, got {len(v)}. " + f'Example: {{"data": [[1, 2], [3, 4]]}}' + ) + + # Check each row has exactly 2 columns + for i, row in enumerate(v): + if len(row) != 2: + raise ValueError( + f"Row {i} has {len(row)} columns, expected exactly 2. " + f'Example: {{"data": [[1, 2], [3, 4]]}}' + ) + + return v +``` + +This example (from `workers/cpu/__init__.py:14-30`) validates: +1. Data is not empty +2. At least 2 rows (prevents NaN in statistics) +3. Each row has exactly 2 columns + +### Validation in FastAPI Router + +Connect request models to endpoints: + +```python +from fastapi import APIRouter +from pydantic import BaseModel, field_validator + +router = APIRouter() + +class DataRequest(BaseModel): + data: list[list[int]] + + @field_validator("data") + @classmethod + def validate_structure(cls, v): + # Custom validation logic + return v + +@router.post("/data") +async def process_endpoint(request: DataRequest): + """FastAPI validates request automatically.""" + result = await process_data({"data": request.data}) + return result +``` + +### Testing Validation + +Valid requests: +```bash +curl -X POST http://localhost:8888/cpu/data \ + -H "Content-Type: application/json" \ + -d '{"data": [[1, 2], [3, 4], [5, 6]]}' +``` + +Invalid requests return 422 with helpful errors: + +```bash +# Too few rows +curl -X POST http://localhost:8888/cpu/data \ + -H "Content-Type: application/json" \ + -d '{"data": [[1, 2]]}' + +# Response: +{ + "detail": [ + { + "type": "value_error", + "msg": "Value error, Need at least 2 rows to compute statistics, got 1. Example: {\"data\": [[1, 2], [3, 4]]}" + } + ] +} +``` + +```bash +# Wrong column count +curl -X POST http://localhost:8888/cpu/data \ + -H "Content-Type: application/json" \ + -d '{"data": [[1, 2, 3], [4, 5, 6]]}' + +# Response: +{ + "detail": [ + { + "type": "value_error", + "msg": "Value error, Row 0 has 3 columns, expected exactly 2. Example: {\"data\": [[1, 2], [3, 4]]}" + } + ] +} +``` + +### Validation Best Practices + +**1. Validate early** - At the API layer, not in worker functions + +```python +# ✅ GOOD - Validation in Pydantic model +class DataRequest(BaseModel): + data: list[list[int]] + + @field_validator("data") + @classmethod + def validate_data(cls, v): + # Validation logic here + return v + +@router.post("/process") +async def endpoint(request: DataRequest): + result = await worker(request.data) # Already validated + return result + +# ❌ BAD - Validation in worker function +@remote(resource_config=config) +async def worker(input_data: dict): + data = input_data["data"] + if not data or not all(len(row) == 2 for row in data): + return {"status": "error", "error": "Invalid data"} + # Process data... +``` + +**2. Provide helpful error messages** + +```python +# ✅ GOOD - Clear, actionable message +raise ValueError( + f"Row {i} has {len(row)} columns, expected exactly 2. " + f'Example: {{"data": [[1, 2], [3, 4]]}}' +) + +# ❌ BAD - Vague message +raise ValueError("Invalid data format") +``` + +**3. Validate constraints, not just types** + +```python +from pydantic import BaseModel, field_validator, Field + +class ImageRequest(BaseModel): + width: int = Field(gt=0, le=4096) # 1-4096 + height: int = Field(gt=0, le=4096) # 1-4096 + quality: int = Field(ge=1, le=100) # 1-100 + + @field_validator("width", "height") + @classmethod + def validate_dimensions(cls, v): + if v % 8 != 0: + raise ValueError(f"Dimension must be divisible by 8, got {v}") + return v +``` + +**4. Use multiple validators for complex validation** + +```python +class DataRequest(BaseModel): + data: list[list[float]] + normalize: bool = False + + @field_validator("data") + @classmethod + def validate_not_empty(cls, v): + if not v: + raise ValueError("Data cannot be empty") + return v + + @field_validator("data") + @classmethod + def validate_dimensions(cls, v): + # Check all rows have same length + if len(set(len(row) for row in v)) > 1: + raise ValueError("All rows must have same length") + return v +``` + +### Common Validation Patterns + +**Range validation:** +```python +from pydantic import Field + +class Request(BaseModel): + temperature: float = Field(ge=0.0, le=1.0) # 0.0 to 1.0 + max_tokens: int = Field(gt=0, le=4096) # 1 to 4096 +``` + +**String validation:** +```python +from pydantic import field_validator +import re + +class TextRequest(BaseModel): + text: str + + @field_validator("text") + @classmethod + def validate_text(cls, v): + if len(v) < 10: + raise ValueError("Text must be at least 10 characters") + if len(v) > 10000: + raise ValueError("Text must not exceed 10,000 characters") + return v +``` + +**List validation:** +```python +class BatchRequest(BaseModel): + items: list[str] + + @field_validator("items") + @classmethod + def validate_batch_size(cls, v): + if len(v) == 0: + raise ValueError("Batch cannot be empty") + if len(v) > 100: + raise ValueError("Batch size cannot exceed 100 items") + return v +``` + +**Enum validation:** +```python +from enum import Enum +from pydantic import BaseModel + +class OutputFormat(str, Enum): + JSON = "json" + CSV = "csv" + PARQUET = "parquet" + +class ExportRequest(BaseModel): + data: list[dict] + format: OutputFormat # Only accepts "json", "csv", or "parquet" +``` + +### Resources + +- [Pydantic Documentation](https://docs.pydantic.dev/) +- [FastAPI Request Validation](https://fastapi.tiangolo.com/tutorial/body/) +- [Field Validators Guide](https://docs.pydantic.dev/latest/concepts/validators/) + +## Version Constraints + +### Exact Version (==) +```python +"torch==2.1.0" # Exactly 2.1.0 +``` +**Use when:** You need reproducible builds + +### Minimum Version (>=) +```python +"Pillow>=10.0.0" # 10.0.0 or higher +``` +**Use when:** You need specific features introduced in a version + +### Maximum Version (<) +```python +"numpy<2.0.0" # Below 2.0.0 +``` +**Use when:** Avoiding breaking changes + +### Compatible Release (~=) +```python +"requests~=2.31.0" # >=2.31.0, <2.32.0 +``` +**Use when:** You want patch updates but not minor updates + +### Latest Version +```python +"pandas" # Latest available +``` +**Use when:** You always want the newest version (not recommended for production) + +## Common Dependencies + +### ML/AI +```python +dependencies=[ + "torch==2.1.0", + "transformers>=4.35.0", + "diffusers", + "accelerate", + "safetensors", +] +``` + +### Data Science +```python +dependencies=[ + "pandas==2.1.3", + "numpy==1.26.2", + "scipy>=1.11.0", + "matplotlib", + "scikit-learn", +] +``` + +### Computer Vision +```python +dependencies=["opencv-python", "Pillow"] +system_dependencies=["libgl1", "libglib2.0-0"] +``` + +### Audio Processing +```python +dependencies=["librosa", "soundfile"] +system_dependencies=["ffmpeg", "libsndfile1"] +``` + +### NLP +```python +dependencies=[ + "transformers>=4.35.0", + "tokenizers", + "sentencepiece", + "spacy", +] +``` + +## System Dependencies + +Common apt packages: + +| Package | Purpose | +|---------|---------| +| `ffmpeg` | Video/audio processing | +| `libgl1` | OpenCV requirement | +| `graphviz` | Graph visualization | +| `libsndfile1` | Audio file I/O | +| `git` | Git operations | +| `wget` | File downloads | + +Example: +```python +system_dependencies=["ffmpeg", "libgl1", "wget"] +``` + +## Best Practices + +### 1. Pin Versions for Production + +```python +# ✅ GOOD - Reproducible +dependencies=[ + "torch==2.1.0", + "transformers==4.35.2", + "numpy==1.26.2", +] + +# ❌ BAD - Unpredictable +dependencies=[ + "torch", # Version changes over time + "transformers", + "numpy", +] +``` + +### 2. Minimize Dependencies + +```python +# ✅ GOOD - Only what's needed +@remote( + dependencies=["requests"] # Just one package +) +async def fetch_data(url: str): + import requests + return requests.get(url).json() + +# ❌ BAD - Unnecessary bloat +@remote( + dependencies=[ + "requests", + "pandas", # Not used + "numpy", # Not used + "scipy", # Not used + ] +) +async def fetch_data(url: str): + import requests + return requests.get(url).json() +``` + +### 3. Test Dependency Compatibility + +```bash +# Test locally first +python -m workers.gpu.endpoint +python -m workers.cpu.endpoint +``` + +### 4. Document Dependencies + +```python +@remote( + resource_config=config, + dependencies=[ + "torch==2.1.0", # GPU operations + "Pillow>=10.0.0", # Image processing + "requests", # API calls + ] +) +async def process_image(data: dict): + """Process image with PyTorch and Pillow.""" + pass +``` + +## Troubleshooting + +### Import Error + +``` +ModuleNotFoundError: No module named 'torch' +``` + +**Solution:** Add to dependencies: +```python +dependencies=["torch"] +``` + +### Version Conflict + +``` +ERROR: Cannot install torch==2.1.0 and torchvision==0.16.0 +because these package versions have conflicting dependencies. +``` + +**Solution:** Check compatibility matrix, adjust versions: +```python +dependencies=[ + "torch==2.1.0", + "torchvision==0.16.0+cu121", # Compatible CUDA version +] +``` + +### System Package Missing + +``` +ImportError: libGL.so.1: cannot open shared object file +``` + +**Solution:** Add system dependency: +```python +system_dependencies=["libgl1"] +``` + +### Slow Cold Start + +Dependencies take long to install? + +**Solutions:** +1. Minimize dependencies +2. Use custom Docker image (advanced) +3. Keep workers warm (workersMin=1) + +## Cold Start Times + +| Dependencies | Cold Start Time | +|-------------|-----------------| +| None | ~5-10 seconds | +| Small (1-2 packages) | ~15-30 seconds | +| Medium (3-5 packages) | ~30-60 seconds | +| Large (torch, transformers) | ~60-120 seconds | + +## Requirements.txt + +For local development, create `requirements.txt`: + +```txt +tetra_rp +torch==2.1.0 +transformers==4.35.2 +Pillow>=10.0.0 +numpy==1.26.2 +``` + +**Note:** Worker dependencies in `@remote` decorator are deployed automatically. `requirements.txt` is for local development only. + +## Advanced: Custom Docker Images + +For complex dependencies, consider custom images: + +```python +from tetra_rp import ServerlessEndpoint + +custom_config = ServerlessEndpoint( + name="custom_image_worker", + dockerImage="myregistry/my-image:v1.0", + gpuIds=["NVIDIA GeForce RTX 4090"], +) + +@remote(resource_config=custom_config) +async def process(data: dict): + # All dependencies pre-installed in image + pass +``` + +See [02_ml_inference/04_custom_images](../../02_ml_inference/04_custom_images/) for details. + +## Next Steps + +- **02_ml_inference** - Deploy real ML models +- **03_advanced_workers** - Caching and optimization +- **04_scaling_performance** - Production patterns + +## Resources + +- [PyPI Package Index](https://pypi.org/) +- [Ubuntu Package Search](https://packages.ubuntu.com/) +- [Runpod Docker Images](https://github.com/runpod/containers) diff --git a/01_getting_started/04_dependencies/main.py b/01_getting_started/04_dependencies/main.py new file mode 100644 index 0000000..e2b8628 --- /dev/null +++ b/01_getting_started/04_dependencies/main.py @@ -0,0 +1,61 @@ +import logging +import os + +from fastapi import FastAPI +from workers.cpu import cpu_router +from workers.gpu import gpu_router + +logger = logging.getLogger(__name__) + + +app = FastAPI( + title="Dependency Management Examples", + description="Learn how to manage Python and system dependencies in Flash workers", + version="0.1.0", +) + +app.include_router(gpu_router, prefix="/gpu", tags=["GPU Workers"]) +app.include_router(cpu_router, prefix="/cpu", tags=["CPU Workers"]) + + +@app.get("/", tags=["Info"]) +def home(): + return { + "message": "Flash Dependency Management Examples", + "description": "Examples of Python and system dependency management", + "docs": "/docs", + "examples": { + "ml_deps": "POST /gpu/ml-deps - ML dependencies (torch, pillow, numpy)", + "system_deps": "POST /gpu/system-deps - System dependencies (ffmpeg, libgl1)", + "data_deps": "POST /cpu/data - Data science dependencies (pandas, scipy)", + "minimal": "POST /cpu/minimal - No dependencies (fastest)", + }, + "concepts": [ + "Version pinning (torch==2.1.0)", + "Version constraints (>=, <, ~=)", + "System packages via apt", + "Minimal dependencies for fast cold start", + ], + } + + +@app.get("/health", tags=["Info"]) +def health(): + return { + "status": "healthy", + "workers": { + "ml_deps": "GPU worker with torch, pillow, numpy", + "system_deps": "GPU worker with ffmpeg, libgl1", + "data_deps": "CPU worker with pandas, scipy", + "minimal": "CPU worker with no dependencies", + }, + } + + +if __name__ == "__main__": + import uvicorn + + port = int(os.getenv("PORT", 8888)) + logger.info(f"Starting Dependency Management server on port {port}") + + uvicorn.run(app, host="0.0.0.0", port=port) diff --git a/01_getting_started/04_dependencies/pyproject.toml b/01_getting_started/04_dependencies/pyproject.toml new file mode 100644 index 0000000..616b7ac --- /dev/null +++ b/01_getting_started/04_dependencies/pyproject.toml @@ -0,0 +1,9 @@ +[project] +name = "04_dependencies" +version = "0.1.0" +description = "Examples of managing Python and system dependencies in Flash workers" +requires-python = ">=3.9" +dependencies = [ + "tetra_rp", + "torch", +] diff --git a/01_getting_started/04_dependencies/requirements.txt b/01_getting_started/04_dependencies/requirements.txt new file mode 100644 index 0000000..415dc57 --- /dev/null +++ b/01_getting_started/04_dependencies/requirements.txt @@ -0,0 +1,2 @@ +tetra_rp +torch diff --git a/01_getting_started/04_dependencies/workers/__init__.py b/01_getting_started/04_dependencies/workers/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/01_getting_started/04_dependencies/workers/cpu/__init__.py b/01_getting_started/04_dependencies/workers/cpu/__init__.py new file mode 100644 index 0000000..180925f --- /dev/null +++ b/01_getting_started/04_dependencies/workers/cpu/__init__.py @@ -0,0 +1,50 @@ +from fastapi import APIRouter +from pydantic import BaseModel, field_validator + +from .endpoint import minimal_process, process_data + +cpu_router = APIRouter() + + +class DataRequest(BaseModel): + """Request model for data processing.""" + + data: list[list[int]] + + @field_validator("data") + @classmethod + def validate_two_columns(cls, v): + if not v: + raise ValueError("Data cannot be empty") + if len(v) < 2: + raise ValueError( + f"Need at least 2 rows to compute statistics, got {len(v)}. " + f'Example: {{"data": [[1, 2], [3, 4]]}}' + ) + for i, row in enumerate(v): + if len(row) != 2: + raise ValueError( + f"Row {i} has {len(row)} columns, expected exactly 2. " + f'Example: {{"data": [[1, 2], [3, 4]]}}' + ) + return v + + +class TextRequest(BaseModel): + """Request model for text processing.""" + + text: str + + +@cpu_router.post("/data") +async def data_endpoint(request: DataRequest): + """Test worker with data science dependencies (pandas, numpy, scipy).""" + result = await process_data({"data": request.data}) + return result + + +@cpu_router.post("/minimal") +async def minimal_endpoint(request: TextRequest): + """Test worker with NO dependencies (fastest cold start).""" + result = await minimal_process({"text": request.text}) + return result diff --git a/01_getting_started/04_dependencies/workers/cpu/endpoint.py b/01_getting_started/04_dependencies/workers/cpu/endpoint.py new file mode 100644 index 0000000..9bd2b1f --- /dev/null +++ b/01_getting_started/04_dependencies/workers/cpu/endpoint.py @@ -0,0 +1,135 @@ +from tetra_rp import CpuInstanceType, CpuLiveServerless, remote + +# Worker with data science dependencies +data_config = CpuLiveServerless( + name="01_04_deps_data", + instanceIds=[CpuInstanceType.CPU3G_2_8], + workersMin=0, + workersMax=5, +) + +# Worker with minimal dependencies +minimal_config = CpuLiveServerless( + name="01_04_deps_minimal", + instanceIds=[CpuInstanceType.CPU3G_2_8], + workersMin=0, + workersMax=5, +) + + +@remote( + resource_config=data_config, + dependencies=[ + "pandas==2.1.3", + "numpy==1.26.2", + "scipy>=1.11.0", + "matplotlib", + ], +) +async def process_data(input_data: dict) -> dict: + """ + Worker with data science dependencies. + + Common data science stack: + - pandas: Data manipulation + - numpy: Numerical operations + - scipy: Scientific computing + - matplotlib: Plotting + """ + from datetime import datetime + + import matplotlib + import numpy as np + import pandas as pd + import scipy + + data = input_data.get("data", [[1, 2], [3, 4], [5, 6]]) + + # Create DataFrame and compute statistics + df = pd.DataFrame(data, columns=["A", "B"]) + stats = { + "mean": df.mean().to_dict(), + "std": df.std().to_dict(), + "sum": df.sum().to_dict(), + } + + # Numpy operation + arr = np.array(data) + numpy_result = { + "shape": arr.shape, + "mean": float(arr.mean()), + } + + versions = { + "pandas": pd.__version__, + "numpy": np.__version__, + "scipy": scipy.__version__, + "matplotlib": matplotlib.__version__, + } + + return { + "status": "success", + "stats": stats, + "numpy_result": numpy_result, + "versions": versions, + "timestamp": datetime.now().isoformat(), + } + + +@remote(resource_config=minimal_config) # No dependencies! +async def minimal_process(input_data: dict) -> dict: + """ + Worker with NO external dependencies. + + Benefits: + - Fastest cold start + - Smallest container size + - No dependency conflicts + - Best for simple operations + """ + import re + from datetime import datetime + + text = input_data.get("text", "") + + # Built-in operations only + word_count = len(text.split()) + char_count = len(text) + uppercase_count = sum(1 for c in text if c.isupper()) + + # JSON manipulation + result = { + "text_analysis": { + "word_count": word_count, + "char_count": char_count, + "uppercase_count": uppercase_count, + "has_numbers": bool(re.search(r"\d", text)), + }, + "python_version": f"3.{__import__('sys').version_info.minor}", + "timestamp": datetime.now().isoformat(), + } + + return { + "status": "success", + "result": result, + "message": "No external dependencies needed!", + } + + +if __name__ == "__main__": + import asyncio + + from dotenv import find_dotenv, load_dotenv + + load_dotenv(find_dotenv()) # Find and load root .env file + + async def test(): + print("\n=== Testing Data Dependencies ===") + data_result = await process_data({"data": [[1, 2], [3, 4], [5, 6]]}) + print(f"Result: {data_result}\n") + + print("=== Testing Minimal Dependencies ===") + minimal_result = await minimal_process({"text": "Hello World 123"}) + print(f"Result: {minimal_result}\n") + + asyncio.run(test()) diff --git a/01_getting_started/04_dependencies/workers/gpu/__init__.py b/01_getting_started/04_dependencies/workers/gpu/__init__.py new file mode 100644 index 0000000..c47de1e --- /dev/null +++ b/01_getting_started/04_dependencies/workers/gpu/__init__.py @@ -0,0 +1,19 @@ +from fastapi import APIRouter + +from .endpoint import process_with_ml_libs, process_with_system_deps + +gpu_router = APIRouter() + + +@gpu_router.post("/ml-deps") +async def ml_deps_endpoint(): + """Test worker with ML dependencies (torch, pillow, numpy).""" + result = await process_with_ml_libs({}) + return result + + +@gpu_router.post("/system-deps") +async def system_deps_endpoint(): + """Test worker with system dependencies (ffmpeg, libgl1).""" + result = await process_with_system_deps({}) + return result diff --git a/01_getting_started/04_dependencies/workers/gpu/endpoint.py b/01_getting_started/04_dependencies/workers/gpu/endpoint.py new file mode 100644 index 0000000..7fd6148 --- /dev/null +++ b/01_getting_started/04_dependencies/workers/gpu/endpoint.py @@ -0,0 +1,125 @@ +from tetra_rp import GpuGroup, LiveServerless, remote + +# Worker with ML dependencies (versioned) +ml_config = LiveServerless( + name="01_04_deps_ml", + gpus=[GpuGroup.ADA_24], + workersMin=0, + workersMax=2, +) + +# Worker with system dependencies +system_deps_config = LiveServerless( + name="01_04_deps_system", + gpus=[GpuGroup.ADA_24], + workersMin=0, + workersMax=2, +) + + +@remote( + resource_config=ml_config, + dependencies=[ + "torch==2.1.0", # Pin specific version + "torchvision", + "Pillow>=10.0.0", # Minimum version + "numpy<2.0.0", # Maximum version constraint + ], +) +async def process_with_ml_libs(input_data: dict) -> dict: + """ + Worker with versioned Python dependencies. + + Best practices: + - Pin exact versions for reproducibility (torch==2.1.0) + - Use >= for minimum versions (Pillow>=10.0.0) + - Use < to avoid breaking changes (numpy<2.0.0) + """ + from datetime import datetime + + import numpy as np + import torch + import torchvision + from PIL import Image + + # Show installed versions + versions = { + "torch": torch.__version__, + "torchvision": torchvision.__version__, + "pillow": Image.__version__, + "numpy": np.__version__, + } + + # Simple tensor operation to verify GPU + if torch.cuda.is_available(): + tensor = torch.randn(100, 100, device="cuda") + result = tensor.sum().item() + else: + result = "No GPU available" + + return { + "status": "success", + "message": "ML dependencies loaded successfully", + "versions": versions, + "gpu_test": result, + "timestamp": datetime.now().isoformat(), + } + + +@remote( + resource_config=system_deps_config, + dependencies=["opencv-python", "requests"], + system_dependencies=["ffmpeg", "libgl1"], # System packages via apt +) +async def process_with_system_deps(input_data: dict) -> dict: + """ + Worker with system-level dependencies. + + system_dependencies installs via apt-get: + - ffmpeg: Video/audio processing + - libgl1: OpenCV requirement + """ + import subprocess + from datetime import datetime + + import cv2 + + # Check FFmpeg installation + try: + ffmpeg_version = ( + subprocess.check_output(["ffmpeg", "-version"], stderr=subprocess.STDOUT) + .decode() + .split("\n")[0] + ) + except Exception as e: + ffmpeg_version = f"Error: {e}" + + # Check OpenCV (requires libgl1) + opencv_version = cv2.__version__ + + return { + "status": "success", + "message": "System dependencies available", + "opencv_version": opencv_version, + "ffmpeg_version": ffmpeg_version, + "timestamp": datetime.now().isoformat(), + } + + +if __name__ == "__main__": + import asyncio + + from dotenv import find_dotenv, load_dotenv + + load_dotenv(find_dotenv()) # Find and load root .env file + + async def test(): + print("\n=== Testing ML Dependencies ===") + ml_result = await process_with_ml_libs({}) + print(f"Result: {ml_result}\n") + + print("=== Testing System Dependencies ===") + sys_result = await process_with_system_deps({}) + print(f"Result: {sys_result}\n") + + asyncio.run(test()) diff --git a/01_getting_started/README.md b/01_getting_started/README.md index c130139..99cebfe 100644 --- a/01_getting_started/README.md +++ b/01_getting_started/README.md @@ -37,14 +37,23 @@ Combining GPU and CPU workers in a single application. - Load balancing between worker types - Common architecture patterns -### 04_dependencies _(coming soon)_ +### [04_dependencies](./04_dependencies/) Managing Python packages and system dependencies. **What you'll learn:** -- Specifying Python dependencies in `@remote` -- Installing system packages (ffmpeg, etc.) -- Managing dependency versions -- Debugging dependency issues +- Python dependency versioning and constraints +- System package installation (ffmpeg, libgl1) +- Input validation with Pydantic field validators +- Version constraints (==, >=, <, ~=) +- Minimizing cold start time +- Best practices for reproducible builds + +**Concepts:** +- `dependencies` parameter for Python packages +- `system_dependencies` parameter for apt packages +- Version pinning for reproducibility +- Pydantic `@field_validator` for request validation +- Dependency optimization strategies ## Learning Path diff --git a/README.md b/README.md index 35dfead..edf9785 100644 --- a/README.md +++ b/README.md @@ -54,10 +54,10 @@ For detailed development instructions, see [DEVELOPMENT.md](./DEVELOPMENT.md). ### 01 - Getting Started Learn the fundamentals of Flash applications. -- **[01_hello_world](./sample_getting_started/)** - Simplest GPU and CPU workers with FastAPI +- **[01_hello_world](./01_getting_started/01_hello_world/)** - The simplest Flash application with GPU workers - 02_cpu_worker - CPU-only worker example _(coming soon)_ - 03_mixed_workers - Combining GPU and CPU workers _(coming soon)_ -- 04_dependencies - Managing Python and system dependencies _(coming soon)_ +- **[04_dependencies](./01_getting_started/04_dependencies/)** - Managing Python and system dependencies ### 02 - ML Inference Deploy machine learning models as APIs. diff --git a/pyproject.toml b/pyproject.toml index b1dadd4..5e3f055 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -6,6 +6,7 @@ readme = "README.md" requires-python = ">=3.9" dependencies = [ "tetra-rp", + "torch", ] [dependency-groups]