Feat #576: accept NumPy array input in batch_image_detection for MDv5 and RT-DETR - #660
Open
proofbyhuman wants to merge 1 commit into
Open
Conversation
…n for MDv5 and RT-DETR batch_image_detection already accepted a list of NumPy arrays in YOLOV8Base, but the other detector backends still required the images to exist on disk, forcing in-memory pipelines (images stored as binary columns in Parquet, PySpark/Dask workflows) to write intermediate JPEG files just to run detection. YOLOV5Base now has the same "handle numpy array input" branch as YOLOV8Base: it batches the arrays, stacks them with the MegaDetector_v5_Transform already used by single_image_detection, and rescales the boxes using each array's own shape. RTDETRApacheBase already iterated one image at a time, so instead of duplicating the loop its image source became a lazy generator: arrays go through Image.fromarray, while the directory case keeps using DetectionImageFolder and opens each file inside the loop exactly as before. In both backends the first parameter was renamed data_path -> data_source to match the YOLOV8Base signature, array inputs get the index as img_id, and there is one result entry per input image. Closes microsoft#576 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuerpo del PR
Summary
batch_image_detectionalready accepts a list of NumPy arrays inYOLOV8Base, but the other detectorbackends still require images on disk. This PR brings the same capability to
YOLOV5Base(MegaDetectorV5) and
RTDETRApacheBase(MegaDetectorV6 Apache), so in-memory pipelines — images storedas binary columns in Parquet, PySpark/Dask workflows — no longer need to write intermediate JPEG files
just to run detection.
Closes #576.
Changes made
PytorchWildlife/models/detection/ultralytics_based/yolov5_base.py# Handle numpy array inputbranch that mirrors the existing one inyolov8_base.py:batches the arrays, stacks them with the same
MegaDetector_v5_Transformalready used bysingle_image_detection, runs NMS withdet_conf_thres, and rescales boxes withscale_boxesusing each array's own
shape[:2].data_path→data_sourceto match theYOLOV8Basesignature.PytorchWildlife/models/detection/rtdetr_apache/rtdetr_apache_base.pysource a lazy generator: NumPy arrays become PIL images via
Image.fromarray, and the directory casekeeps using
DetectionImageFolderand opens each file inside the loop exactly as before (no eagerloading of the whole folder).
data_path→data_sourcerename.In both backends, array inputs get the index as
img_id("0","1", …) and one result entry perinput image, matching what
YOLOV8Basealready does.Why this works
The transforms in both backends already accepted in-memory images —
single_image_detectionhas takenndarrayinput inyolov5_base.pyandrtdetr_apache_base.pyfor a long time. The only thing missingwas a path into the batch method, so this reuses the existing preprocessing rather than adding a second
code path for it.
Testing
Tested locally on CPU with the real pretrained weights, comparing both input modes over the same four
images. The images are deliberately of different, non-square sizes — 800×600, 640×480, 1024×768 and one
portrait 600×800 — so that any width/height mix-up in the box rescaling would show up as a mismatch:
md_v5a.0.0.pt): ranbatch_image_detection(folder), thenbatch_image_detection(list_of_arrays)with the same images loaded vianp.array(Image.open(p).convert("RGB")). Both passes returned 4 results with 16 detections in total,and
xyxy,confidence,class_idandnormalized_coordsmatched exactly.MDV6-apa-rtdetr-c.pth): same comparison, same outcome — identicaldetections from disk and from arrays.
Out of scope / notes for maintainers
yolo_mit_base.pyis not included. It setscfg.task.data.sourceto the path and reloads themodel on every call, so array support there is a different change. Happy to send a follow-up PR.
data_path→data_sourcerename. No caller inside the repo passes it as a keyword (all arepositional), and it aligns the three backends, but it is technically a breaking change for anyone
calling
batch_image_detection(data_path=...)externally. Tell me if you'd rather keep the old name.if pred.size(0) == 0: continue,so images with no detections are dropped from the results, while
yolov8_base.pykeeps them. The newarray branch keeps them (consistent with YOLOv8). I did not change the directory branch, but this
inconsistency looks related to Mismatch in number of results between detector and classifier #610 and I can open a separate issue/PR if useful.
rtdetr_apache_base.py(unchanged by this PR, but confirmed while testing).normalized_coordsis computed fromsize = torch.tensor([w, h]), sox / size[1]divides x by theheight and
y / size[0]divides y by the width. On the non-square test images the normalised valuescame out as high as 1.34 — exactly the 800/600 aspect ratio — instead of staying within
[0, 1].This affects the Timelapse-compatible output for any non-square image. I kept the behaviour untouched
so this PR stays focused; glad to fix it in a separate PR if you want.
AI assistance
Author: @proofbyhuman, with AI assistance from Claude Code (Claude Fable 5) for exploring the codebase
and drafting the change. I reviewed and tested everything myself, and the implementation deliberately
follows the pattern already present in
yolov8_base.pyrather than introducing a new one.Thanks a lot for taking the time to review this — happy to rework any part of it.