Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- run: ruff check .
- run: python -m build
- run: pytest
- run: python tools/validate_results.py --config configs/contest/effiped-tier1.yaml
- run: python tools/validate_results.py --config configs/system/effiped-tier1.yaml
- run: python tools/validate_media.py
- run: python tools/validate_release.py

Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ runs/
*.mov
*.mkv
*.webm
!docs/media/pdestre/effiped-demo.webm
!docs/media/pdestre/multi-camera-tracking.webm
!docs/media/pdestre/cross-camera-matches.webm
tmp/
*.log
*.aux
Expand Down
16 changes: 0 additions & 16 deletions AWARD.md

This file was deleted.

8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Contributing

Keep contributions focused on the contest system, local identity-review workflow, or verified
research evolution. Add tests proportional to the change and run:
Keep contributions focused on the EffiPed model, local identity-review workflow, or verified
descriptor research. Add tests proportional to the change and run:

```bash
ruff check .
Expand All @@ -18,5 +18,5 @@ Do not commit datasets, checkpoints, source videos, runtime crops, secrets, abso
paths, unrelated projects, or unsupported claims. Every P-DESTRE-derived asset must be
non-commercial, attributed, hashed, and listed in `docs/media/ASSET_MANIFEST.json`.

Code contributions are Apache-2.0. Media and third-party material retain their own compatible
terms.
Code contributions are Apache-2.0. Media and third-party material retain their own
compatible terms.
33 changes: 17 additions & 16 deletions MODEL_CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,43 @@
## Summary

EffiPed Tier-1 is a compact joint detection and embedding model built around ConvNeXt V2,
P2/P3 fusion, CenterNet outputs, RoIAlign, four body strips, CoordinateAttention, and a
256-D normalized descriptor. BoT-SORT handles local temporal association; a bounded gallery
ranks cross-camera candidates.
P2/P3 fusion, CenterNet outputs, RoIAlign, four body strips, Coordinate Attention, and a
256-D normalized descriptor. BoT-SORT handles camera-local temporal association; a bounded
gallery ranks cross-camera candidates.

## Intended use

- Non-commercial research and education.
- Reproduction of the documented contest and matched evaluation configurations.
- Reproduction of the documented system and matched descriptor evaluations.
- Local, human-in-the-loop review of user-authorized camera video.
- Portfolio demonstration using precomputed, attributed media.

## Out-of-scope use

- Automated identity decisions or claims of identity.
- Face recognition or biometric identification.
- Public or covert surveillance without a purpose-specific legal, privacy, consent, security,
retention, bias, and human-review assessment.
- Public or covert surveillance without a purpose-specific legal, privacy, consent,
security, retention, bias, and human-review assessment.
- Commercial use while training-data terms remain non-commercial or unresolved.

## Evidence boundary

Verified contest evidence includes 62.8% validation and 61.3% test cross-camera Rank-1 on
P-DESTRE, plus 64.08 MOTA, 74.24 IDF1, and 61.34 HOTA on MOT17 val-half. See `RESULTS.md`
for the exact protocol ledger and the separately labeled poster and post-contest evidence.
Published system evidence includes 62.8% validation and 61.3% test cross-camera Rank-1 on
P-DESTRE, plus 64.08 MOTA, 74.24 IDF1, and 61.34 HOTA on MOT17 val-half. See
`RESULTS.md` for the complete protocol ledger.

## Limitations

Scores are sensitive to occlusion, detector localization, pose, clothing ambiguity, illumination,
camera calibration, time gaps, domain shift, and crowd density. A high cosine similarity is
candidate evidence, not proof that two observations depict the same person. The public
evaluation does not establish performance for other locations, populations, or camera networks.
Scores are sensitive to occlusion, detector localization, pose, clothing ambiguity,
illumination, camera calibration, time gaps, domain shift, and crowd density. A high cosine
similarity is candidate evidence, not proof that two observations depict the same person.
The published evaluation does not establish performance for other locations, populations,
or camera networks.

## Artifact status

The versioned manifest describes `effiped-tier1-contest-v1.pt`, but public checkpoint
distribution is on hold. Runtime construction always sets `pretrained=false` when loading a
checkpoint, avoiding an unnecessary backbone download.
The versioned manifest describes `effiped-tier1-v1.pt`, but public checkpoint distribution
is on hold. Runtime construction always uses `pretrained=false` when loading a checkpoint,
avoiding an unnecessary backbone download.

Maintainer: [Aswanth Raj](https://github.com/aswanth-07)
163 changes: 76 additions & 87 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,90 +1,86 @@
<div align="center">
<img src="docs/social-preview.png" alt="EffiPed — 3rd-Prize multi-camera pedestrian tracking system" width="100%">
<img src="docs/social-preview.png" alt="EffiPed multi-camera pedestrian tracking and identity-review system" width="100%">

# EffiPed

## Multi-Camera Pedestrian Detection, Tracking & Re-Identification using Joint ConvNeXt V2 Architecture

**3rd Prize — Student Innovation Project Contest 2026**
Vertical 1: AI & Intelligent Systems · VIT Vellore SCOPE

By [Aswanth Raj](https://github.com/aswanth-07) · Guide: Sri Preethaa KR
By [Aswanth Raj](https://github.com/aswanth-07)

[![CI](https://github.com/aswanth-07/effiped-multi-camera-tracking/actions/workflows/ci.yml/badge.svg)](https://github.com/aswanth-07/effiped-multi-camera-tracking/actions/workflows/ci.yml)
[![Software: Apache-2.0](https://img.shields.io/badge/software-Apache--2.0-24e6bd)](LICENSE)
[![Media: CC BY-NC-SA 4.0](https://img.shields.io/badge/P--DESTRE_media-CC_BY--NC--SA_4.0-e8a94d)](docs/media/LICENSE.md)
[![Software: Apache-2.0](https://img.shields.io/badge/software-Apache--2.0-22c7b8)](LICENSE)
[![Media: CC BY-NC-SA 4.0](https://img.shields.io/badge/P--DESTRE_media-CC_BY--NC--SA_4.0-70b8ff)](docs/media/LICENSE.md)
</div>

EffiPed is a compact video-intelligence system that detects pedestrians, maintains
camera-local tracks, and ranks cross-camera identity candidates for human review. Its
React investigation console is available as a precomputed browser demo; the same workflow
can connect to local FastAPI/CUDA inference when an authorized checkpoint is available.

> [!IMPORTANT]
> The hosted experience is a precomputed, non-commercial research demonstration. It ranks
> appearance evidence for human review; it does not prove identity. Public model weights are
> withheld while training-data redistribution terms remain unresolved.
> Ranked matches are reviewable appearance evidence, not proof of identity. The hosted
> experience is a precomputed, non-commercial research demonstration. Public model weights
> remain withheld while training-data redistribution terms are unresolved.

## Try the identity-review demo

The hosted UI restores the original PedestrianTracker workflow:

- a synchronized four-camera replay with tracker-rendered boxes;
- indexed query crops and ranked cross-camera candidates;
- camera scope, playback, frame stepping, and detection timelines;
- a second replay containing the archived cross-camera association output;
- responsive desktop and mobile review modes.

```bash
cd apps/web
npm install
npm run dev
```

No synthetic browser boxes are drawn over the footage. The boxes visible in the demo are
the annotations rendered by the original tracking pipeline.

## The contest system
## System

EffiPed shares one compact ConvNeXt V2 feature hierarchy across three connected tasks:
CenterNet-style pedestrian detection, BoT-SORT temporal association, and 256-D part-based
descriptors for cross-camera candidate retrieval. RoIAlign divides each person into four
body strips, CoordinateAttention weights visible evidence, and the analyst UI exposes the
result as reviewable candidates rather than an automated identity verdict.
One ConvNeXt V2 feature hierarchy supports CenterNet-style detection and a 256-D
part-aware identity descriptor. RoIAlign extracts a person feature map, four horizontal
body strips retain local appearance, and Coordinate Attention fuses the visible evidence.
BoT-SORT combines motion, overlap, and appearance for local temporal association; the
gallery then ranks possible cross-camera matches for an analyst.

| Verified contest evidence | Result |
| Evaluation | Result |
|---|---:|
| P-DESTRE validation cross-camera Rank-1 | **62.8%** |
| P-DESTRE test cross-camera Rank-1 | **61.3%** |
| P-DESTRE validation / test detection mAP@0.5 | **90.74% / 88.4%** |
| MOT17 val-half MOTA / IDF1 / HOTA | **64.08 / 74.24 / 61.34** |
| Canonical Tier-1 footprint | **7.78M · ≈18 full-pipeline FPS** |
| EffiPed Tier-1 footprint | **7.78M · ≈18 full-pipeline FPS** |

The submitted poster is preserved as an archived contest artifact with its original
`7.92M / 22 FPS / 62.8%` snapshot. The later canonical registry associates Tier-1 with
`7.78M` parameters and approximately `18 FPS` for the full pipeline. The poster’s
`+16.2 pp` row combined multiple configuration changes and is not presented as a pure
part-only ablation. See [RESULTS.md](RESULTS.md).
Each value has a protocol label in [RESULTS.md](RESULTS.md). The interactive replay is an
application demonstration, not a benchmark run.

## From contest prototype to research
## Architecture

```text
EffiPed contest system
joint detection + four-strip descriptor + multi-camera review
├── PartJDE matched readout study: +6.66 pp validation Rank-1
└── BoxJDE five-fold readout study:
+13.64/+12.94 pp source-level Rank-1/mAP
+13.31/+12.29 pp natural predicted-box
+13.01/+12.00 pp natural end-to-end
```
[![EffiPed end-to-end architecture](docs/architecture/effiped-architecture.svg)](docs/architecture/effiped-architecture.svg)

BoxJDE uses a constructed P-DESTRE per-date ablation, not official Task 4. Its complete
code, evidence, and technical report live in the
[BoxJDE Person Search repository](https://github.com/aswanth-07/boxjde-person-search).
The diagram is also available as an
[editable PowerPoint](docs/architecture/effiped-architecture.pptx).

## Repository map

```text
src/effiped/ installable model, descriptor, tracker, runtime
src/effiped/ installable model, descriptors, tracking, runtime
apps/api/ FastAPI local-GPU service and job lifecycle
apps/web/ React/Vite portfolio + identity-review UI
configs/contest/ contest and matched PartJDE configurations
research/results/ the single evidence fixture
apps/web/ React/Vite identity-review UI and hosted replay
configs/system/ active EffiPed and matched PartJDE configurations
research/results/ single source of truth for published evidence
research/report/ generated technical report
docs/architecture/ editable PowerPoint + site exports
docs/media/ optimized attributed demonstration media
tools/ and tests/ release validation and regression tests
```

## Explore the Vercel-safe demo

```bash
cd apps/web
npm install
npm run dev
docs/architecture/ editable diagram source and web exports
docs/media/ optimized, attributed demonstration media
tools/ and tests/ validation, regression, and release checks
```

The demo supports camera switching, clickable tracks, query selection, confidence-grouped
cross-camera candidates, timeline navigation, and evidence details without uploading video.

## Run live inference locally

Python 3.11 and an NVIDIA GPU are recommended.
Expand All @@ -97,20 +93,15 @@ pip install -e ".[runtime]"
effiped-app
```

Place an authorized checkpoint in `EFFIPED_WEIGHTS_DIR`; the API reports unavailable
weights cleanly when none is present.

Containerized local runtime:
Place an authorized checkpoint in `EFFIPED_WEIGHTS_DIR`. When none is present, the API
reports the model as unavailable without exposing a local filesystem path.

```bash
docker build -t effiped .
docker run --gpus all --rm -p 127.0.0.1:8000:8000 \
-v /authorized/weights:/weights:ro -v effiped-runtime:/runtime effiped
effiped-train --config configs/system/effiped-tier1.yaml
effiped-eval --config configs/system/effiped-tier1.yaml
effiped-demo
```

The loopback-only host mapping keeps the review surface local while the container listens on
its internal interface.

| Variable | Purpose |
|---|---|
| `EFFIPED_WEIGHTS_DIR` | authorized local model artifacts |
Expand All @@ -119,14 +110,6 @@ its internal interface.
| `EFFIPED_MAX_UPLOAD_MB` | per-video upload limit |
| `EFFIPED_ALLOWED_ORIGINS` | comma-separated CORS allowlist |

Commands:

```bash
effiped-train --config configs/contest/effiped-tier1.yaml
effiped-eval --config configs/contest/effiped-tier1.yaml
effiped-demo
```

## Public API

- `GET /api/health`
Expand All @@ -138,22 +121,28 @@ effiped-demo
- `DELETE /api/person-search/jobs/{job_id}`
- `GET /api/assets/{asset_id}`

Deleting a job removes uploaded video and all generated assets. Model metadata is portable
and never exposes workstation paths.
Deleting a job removes uploaded video and generated assets.

## Research connections

The later [BoxJDE Person Search](https://github.com/aswanth-07/boxjde-person-search)
repository isolates the full-person descriptor readout and documents its five-fold
P-DESTRE ablation. It is linked as related research; its code and report are not duplicated
here.

## Reports, architecture, and responsible use
## Licensing and responsible use

- [Editable architecture PowerPoint](docs/architecture/effiped-architecture.pptx)
- [Technical report](docs/report/effiped-technical-report.pdf)
- [Model card](MODEL_CARD.md)
- [Award record](AWARD.md)
- [Data and weight-release audit](DATA_LICENSES.md)
- [Third-party notices](THIRD_PARTY_NOTICES.md)
Original software is © 2026 Aswanth Raj and licensed under Apache-2.0. P-DESTRE-derived
media under `docs/media/pdestre/` is separately licensed as a CC BY-NC-SA 4.0 adaptation
for this non-commercial showcase. The
[asset manifest](docs/media/ASSET_MANIFEST.json) records the source, transformations,
hash, purpose, and license for every derived asset.

Software is © 2026 Aswanth Raj and licensed under Apache-2.0. P-DESTRE-derived media under
`docs/media/pdestre/` is a separate CC BY-NC-SA 4.0 adaptation for this non-commercial
showcase. Its [asset manifest](docs/media/ASSET_MANIFEST.json) records transformations and
hashes. No source videos, datasets, person-level benchmark records, or checkpoints are included.
No dataset, source video, person-level benchmark record, checkpoint, or runtime crop is
included.

[Model card](MODEL_CARD.md) ·
[Data and weight-release audit](DATA_LICENSES.md) ·
[Third-party notices](THIRD_PARTY_NOTICES.md) ·
[P-DESTRE paper](https://arxiv.org/abs/2004.02782) ·
[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)
30 changes: 17 additions & 13 deletions RESULTS.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Results and Protocol Ledger

All website cards, charts, README values, and report tables are generated from
The website, README, report, charts, and tests read the same checked-in fixture:
`research/results/summary.json`.

## 1. Verified contest system
## EffiPed system benchmarks

| Evaluation | Metric | Value |
|---|---|---:|
Expand All @@ -15,17 +15,13 @@ All website cards, charts, README values, and report tables are generated from
| MOT17 val-half Protocol A | IDF1 | 74.24 |
| MOT17 val-half Protocol A | HOTA | 61.34 |

Canonical Tier-1 footprint: 7.78M parameters and approximately 18 FPS for the full tracking
pipeline at 1088×608 on an NVIDIA RTX 5080 Laptop GPU.
EffiPed Tier-1 contains 7.78M parameters and runs at approximately 18 FPS for the complete
tracking pipeline at 1088×608 on an NVIDIA RTX 5080 Laptop GPU.

## 2. Contest submission snapshot
The browser replay is not used to measure these numbers. It is an interactive presentation
of archived detector, tracker, and cross-camera association output.

The submitted poster displayed 7.92M parameters, 22 FPS, and 62.8% cross-camera Rank-1.
That snapshot is preserved as submitted. It is not substituted for the later canonical
Tier-1 measurement. The poster’s +16.2 percentage-point row combined multiple configuration
differences; it is not a pure part-readout ablation.

## 3. Post-contest evolution
## Descriptor research extensions

| Study | Boundary | Rank-1 gain | mAP gain |
|---|---|---:|---:|
Expand All @@ -34,5 +30,13 @@ differences; it is not a pure part-readout ablation.
| BoxJDE | five-fold natural predicted boxes | +13.31 pp | +12.29 pp |
| BoxJDE | five-fold natural end-to-end | +13.01 pp | +12.00 pp |

PartJDE’s separate evaluation reports 7.92M parameters and 27.0 FPS. BoxJDE’s primary
P-DESTRE protocol is a constructed per-date descriptor-readout ablation, not official Task 4.
PartJDE's separate evaluation reports 7.92M parameters and 27.0 FPS. BoxJDE uses a
constructed P-DESTRE per-date descriptor-readout ablation, not official Task 4; its full
evidence and technical report live in the
[BoxJDE repository](https://github.com/aswanth-07/boxjde-person-search).

## Interpretation boundary

Cross-camera similarities rank candidate evidence. They do not establish identity, and the
reported evaluations do not establish behavior for other sites, populations, cameras, or
operating conditions.
2 changes: 1 addition & 1 deletion apps/api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ def models():
@app.post("/api/person-search/jobs", response_model=JobCreateOut)
def create_person_search_job(
files: List[UploadFile] = File(...),
model_key: str = Form("effiped_tier1_contest"),
model_key: str = Form("effiped_tier1"),
decode_thresh: float = Form(0.05),
track_thresh: float = Form(0.25),
max_frames: int = Form(0),
Expand Down
Loading
Loading