Skip to content
NickEinstein1
PROSOPO
Public

About

PROSOPO is a outerworld facial recognition system designed with a focus on vision with accuracy and speed. It integrates state-of-the-art detection and recognition models with advanced fairness-aware techniques to ensure equitable performance across different species.

Topics

bias-mitigationcomputer-vision-pythondemographic-fairnessface-detectionface-embeddingfacial-analysisfacial-recognitionresponsible-aivision-language-model

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

1 fork
Report repository
BranchesTags

Repository files navigation

Face AI

State-of-the-art unbiased facial recognition system.

Overview

Face AI is a comprehensive facial recognition pipeline designed with a focus on fairness and bias mitigation. It integrates state-of-the-art detection and recognition models with advanced fairness-aware techniques to ensure equitable performance across different demographic groups.

Key Features

  • Unified Pipeline: Seamless integration of face detection, alignment, embedding extraction, and recognition.
  • Fairness-First Design: Built-in bias detection and mitigation strategies.
  • Demographic Calibration: Tools to calibrate thresholds per demographic group to ensure equal error rates.
  • Production Ready: Designed for deployment with optimized Docker builds and robust error handling.

Installation

Prerequisites

  • Python 3.9+
  • Docker (optional, for containerized deployment)

Local Setup

  1. Clone the repository:

    git clone https://github.com/NickEinstein1/PROSOPO.git
cd PROSOPO

  2. Create and activate a virtual environment:

    python -m venv venv
source venv/bin/activate

  3. Install dependencies:

    pip install -e .

    For development dependencies:

    pip install -e ".[dev]"

Usage

Basic Example

from face_ai.core.pipeline import FaceAIPipeline

# Initialize the pipeline
pipeline = FaceAIPipeline()

# Detect faces
faces = pipeline.detect_faces("path/to/image.jpg")
print(f"Detected {len(faces)} faces")

# Verify identity
is_same, score = pipeline.verify("photo1.jpg", "photo2.jpg")
print(f"Match: {is_same}, Score: {score}")

Fairness Aware Recognition

from face_ai.core.pipeline import FaceAIPipeline

pipeline = FaceAIPipeline(enable_fairness=True)

# Register identity with metadata
pipeline.register_identity(
    name="user_123",
    images=["user_photo.jpg"],
    metadata={"demographic": "group_a"}
)

# Generate fairness report
report = pipeline.get_fairness_report()
print(report)

Configuration

Runtime settings are loaded from environment variables (edge, API, Docker):

Variable Default Description
FACE_AI_THRESHOLD 0.5 Cosine similarity match threshold
FACE_AI_MODEL_PACK buffalo_l buffalo_l, buffalo_sc (edge), antelopev2 (stronger), custom
FACE_AI_RECOGNITION_ONNX — Custom ArcFace ONNX when pack is custom
FACE_AI_DETECTION_PACK buffalo_l Detector pack used with custom recognition
FACE_AI_INSIGHTFACE_ROOT — InsightFace model zoo cache directory
FACE_AI_DEVICE cpu cpu or cuda
FACE_AI_GALLERY_FUSION max max, mean, or quality_weighted
FACE_AI_STORAGE_BACKEND memory memory or qdrant
FACE_AI_MAX_FACES 10 Max faces per image
FACE_AI_MIN_QUALITY_SCORE 0.35 Min face quality for enrollment
FACE_AI_TARGET_FAR 0.001 Target false positive rate for calibration
FACE_AI_CALIBRATION_PATH auto Per-pack threshold file, or explicit JSON path
FACE_AI_THRESHOLDS_DIR data/thresholds Directory of {pack}.json threshold files
FACE_AI_INTERNAL_BENCHMARK_DIR data/eval/internal Production capture + pairs root
FACE_AI_CAPTURE_BENCHMARK false Save enroll/verify images for internal benchmark
FACE_AI_CAPTURE_BENCHMARK_SECRET — Required (min 16 chars) when capture is enabled; send as X-Face-AI-Capture-Token
FACE_AI_FAIRNESS_DIR — Directory with demographic pairs.csv for /fairness/calibrate
FACE_AI_SECURITY_MODE standard standard or high (fail-closed liveness on enroll/verify)
FACE_AI_PROFILE server server, edge, or high_security
FACE_AI_ENABLE_AUDIT true Log enroll/recognize/verify to audit file
FACE_AI_AUDIT_LOG logs/audit.log Audit JSONL path
FACE_AI_CALIBRATION_DIR — Server directory with pairs.csv for API /calibrate
FACE_AI_REQUIRE_LIVENESS_VERIFY false Require liveness on /verify by default
FACE_AI_LIVENESS_FAIL_CLOSED true Reject when anti-spoof model unavailable
FACE_AI_LIVENESS_MODEL liveness/minifasnet.onnx ONNX path under FACE_AI_MODEL_PATH
FACE_AI_JOBS_DIR data/jobs Async video job storage
FACE_AI_VIDEO_MAX_WORKERS 1 Parallel video job workers
FACE_AI_VIDEO_FRAME_STRIDE 5 Default process-every-Nth-frame
FACE_AI_VIDEO_MAX_UPLOAD_MB 512 Max upload size per video 
from face_ai.core.pipeline import FaceAIPipeline
from face_ai.config import get_settings

pipeline = FaceAIPipeline.from_settings(get_settings())

API endpoints (Phase 2)

Method Path Description
POST /liveness Anti-spoof check per face
POST /verify?require_liveness=true 1:1 verify with liveness gate
POST /calibrate Threshold calibration from FACE_AI_CALIBRATION_DIR
GET /calibration Current calibrated threshold
POST /video/jobs Enqueue one or more videos (async)
GET /video/jobs/{id} Job status / progress
GET /video/jobs/{id}/result Full batch summary when finished
POST /video/analyze Short clip, synchronous
GET /models/packs List recognition packs and active model

Place MiniFASNet ONNX at models/liveness/minifasnet.onnx for silent anti-spoof.

Download liveness model

face-ai download-models liveness
# or
python scripts/download_liveness_model.py
python scripts/download_liveness_model.py --check   # verify SHA256

See models/liveness/README.md for manual download and Docker setup.

Threshold calibration

python benchmarks/calibrate_threshold.py --data-dir data/calibration --output data/calibration.json

For pack ranking on LFW / CFP-FP / an internal set, see Benchmarks below.

Benchmarks

Measured numbers (antelopev2)

From a local run with --write-thresholds (partial LFW + bootstrapped internal set; re-run on full LFW/CFP-FP for publication):

Dataset Pairs EER TAR@FAR=1e-3 Notes
LFW (partial) 152 1.37% 97.3% Full View-2 = 6000 pairs
Internal 458 3.71% 95.9% Replace with production cameras
Mean — 2.54% 96.6% Pack recommendation: antelopev2

Threshold written to data/thresholds/antelopev2.json (FAR=1e-3 operating point ≈ 0.168 on that internal set).

CI runs parser/unit tests on every PR (.github/workflows/ci.yml). Full LFW/CFP-FP/IJB-C is not downloaded in CI (too large); run locally:

# Public LFW
face-ai benchmark --download lfw --lfw-dir data/eval/lfw

# Compare packs + write per-pack thresholds
face-ai benchmark \
  --datasets lfw,cfp_fp,internal \
  --packs buffalo_l,buffalo_sc,antelopev2 \
  --write-thresholds \
  -o data/eval/reports/pack_comparison.json

RFW-style subgroup tables: face_ai.eval.subgroup.subgroup_table (needs demographic-labeled pairs).

Fairness on the match path

Demographic thresholds from BiasMitigation are applied in FaceRecognizer.match():

  1. Probe predicted_ethnicity is filled when attributes are on (appearance proxy + gender).
  2. Enrollment metadata.demographic is used when the probe label is missing.
  3. Calibrated score/threshold via get_fair_match_decision().
export FACE_AI_FAIRNESS_DIR=data/fairness   # pairs.csv: path1,path2,demographic,label
curl -X POST "http://localhost:8000/fairness/calibrate?target_fpr=0.001"

Optional fairlearn / aif360 enrich /fairness summaries when installed (pip install -e ".[fairness]").

Security modes

Mode Env Behavior
standard FACE_AI_SECURITY_MODE=standard Liveness optional on verify
high FACE_AI_SECURITY_MODE=high or FACE_AI_PROFILE=high_security Liveness required on enroll + verify; fail-closed if ONNX missing; min 2 enrollment images 
docker compose --profile secure up face-ai-api-secure

Deployment profiles

Profile Env Effect
server FACE_AI_PROFILE=server Default; Docker uses antelopev2
edge FACE_AI_PROFILE=edge buffalo_sc, det_size 320, CPU
high_security FACE_AI_PROFILE=high_security Same as high security mode

Audit logging: FACE_AI_ENABLE_AUDIT=true (default) writes to FACE_AI_AUDIT_LOG (API enroll/recognize/verify).

Qdrant gallery: docker compose --profile with-qdrant up then FACE_AI_STORAGE_BACKEND=qdrant.

Jobs note: video jobs run in-process. Restarting the API loses in-flight work; for production use an external queue (Redis/RQ/Celery) later.

Recognition model packs

Ship the stronger pack with per-pack thresholds:

# InsightFace ResNet100 (recommended for accuracy)
face-ai download-models pack antelopev2
export FACE_AI_MODEL_PACK=antelopev2
export FACE_AI_CALIBRATION_PATH=auto   # loads data/thresholds/antelopev2.json

# Custom ArcFace ONNX + buffalo_l detector
face-ai download-models recognition w600k_r50
export FACE_AI_MODEL_PACK=custom
export FACE_AI_RECOGNITION_ONNX=models/recognition/w600k_r50.onnx
export FACE_AI_DETECTION_PACK=buffalo_l
# threshold file: data/thresholds/custom_w600k_r50.json

# Calibrate / refresh thresholds
face-ai calibrate-pack --pack antelopev2 --data-dir data/calibration
face-ai benchmark --datasets lfw --packs antelopev2 --write-thresholds
face-ai thresholds

Docker:

docker compose up --build                              # antelopev2 (default)
docker compose --profile strong up face-ai-api-strong  # antelopev2 alias
docker compose --profile secure up face-ai-api-secure  # high-security
docker compose --profile custom up face-ai-api-custom  # custom ONNX
docker compose --profile with-qdrant up                # + Qdrant
Pack Role
buffalo_l Balanced SCRFD + ResNet50 ArcFace (512-d).
buffalo_sc Edge/CPU. Smaller detector + MobileFaceNet.
antelopev2 Default in Docker. ResNet100. Higher accuracy.
custom Your ArcFace/MagFace/AdaFace ONNX + InsightFace detector. 
face-ai packs
face-ai download-models pack buffalo_sc
face-ai download-models pack antelopev2

# Edge
export FACE_AI_PROFILE=edge
# or
export FACE_AI_MODEL_PACK=buffalo_sc

# Stronger official
export FACE_AI_MODEL_PACK=antelopev2

# Custom recognizer (keep buffalo_l for detection)
export FACE_AI_MODEL_PACK=custom
export FACE_AI_RECOGNITION_ONNX=models/recognition/w600k_r50.onnx
export FACE_AI_DETECTION_PACK=buffalo_l

Place custom graphs under models/recognition/ (see that folder’s README). /health and GET /models/packs report the active pack.

Testing

Run the test suite using pytest:

pytest

Batch video (async)

Submit long or multiple videos without blocking the API:

# Local batch (writes JSON)
face-ai process-videos clip1.mp4 clip2.mp4 --stride 5 -o data/video_results

# API: enqueue, then poll
curl -X POST http://localhost:8000/video/jobs \
  -F "files=@clip1.mp4" -F "files=@clip2.mp4" \
  -F "frame_stride=5" -F "enable_liveness=false"

curl http://localhost:8000/video/jobs/{job_id}
curl http://localhost:8000/video/jobs/{job_id}/result
Method Path Description
POST /video/jobs Enqueue one or more videos
GET /video/jobs List recent jobs
GET /video/jobs/{id} Job status / progress
GET /video/jobs/{id}/result Full summary when finished
POST /video/jobs/{id}/cancel Cooperative cancel
POST /video/analyze Small clips, synchronous

Jobs persist under FACE_AI_JOBS_DIR (default data/jobs). Inference runs in a background worker (FACE_AI_VIDEO_MAX_WORKERS, default 1).

Docker Support

Build and run the API server (MiniFASNet liveness model is downloaded at build time):

docker compose up --build

The image includes:

  • Full Python dependencies (InsightFace, ONNX Runtime, FastAPI)
  • MiniFASNet ONNX at /app/models/liveness/minifasnet.onnx
  • Entrypoint that re-downloads liveness weights if the models volume is empty

Check deployment health:

curl http://localhost:8000/health

Development with hot reload:

docker compose --profile dev up face-ai-dev

About

PROSOPO is a outerworld facial recognition system designed with a focus on vision with accuracy and speed. It integrates state-of-the-art detection and recognition models with advanced fairness-aware techniques to ensure equitable performance across different species.

Topics

bias-mitigationcomputer-vision-pythondemographic-fairnessface-detectionface-embeddingfacial-analysisfacial-recognitionresponsible-aivision-language-model

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages