Skip to content

feat: add depth estimation task implementation - #1552

Merged
barhanc merged 9 commits into
mainfrom
@bh/depth-anything
Oct 20, 2026
Merged

barhanc merged 9 commits into
mainfrom
@bh/depth-anything

Conversation

@barhanc

@barhanc barhanc commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Description

This PR introduces new depth estimation task implementation along with a test suite and a simple example screen in computer vision app to test the new feature end-to-end. Alongside it we upload 3 depth estimation models to SWM HuggingFace repo to use with this new task with different performance/quality tradeoffs; all accessible in the new collection https://huggingface.co/collections/software-mansion/depth-estimation.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

Testing instructions

Screenshots

Related issues

Closes #1372

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

@barhanc barhanc self-assigned this Oct 19, 2026
@barhanc barhanc added the feature PRs that implement a new feature label Oct 19, 2026
@barhanc
barhanc marked this pull request as ready for review October 19, 2026 15:39
@barhanc
barhanc requested a review from msluszniak October 19, 2026 15:39
Comment thread docs/docs/02-extensions/computer-vision/12-depth-estimation.md

@msluszniak msluszniak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HF configs fail the spec schema:

  1. family: depth-anything-v2/-v3 don't match ^[a-z0-9_]+$, should be depth_anything_v2/_v3.
  2. capabilities: depth-estimation isn't in the enum. Neither are gemma-4-multimodal's audio-text-to-text/image-text-to-text, so let's add all three to the spec.
  3. Variants carry quantized/default, which the schema disallows (also in the blazeface and rfdetr-keypoint Vulkan configs).

Cards:
4. ZipDepth's "Repository structure" has no sizes on the config.json lines.
5. ZipDepth Core ML is fp32, while other CV Core ML builds are fp16 (fp32 never reaches the ANE). If fp16 hurts accuracy, please note that on the card.
6. Please add an "XNNPACK int8" section with quality numbers to both Depth Anything cards, like the other int8 repos have.

@msluszniak msluszniak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code suggestions:

  1. download-libs.js: the FEATURE_MAP comment names only Depth Anything V2; V3 and ZipDepth ship the same three backends.
  2. Docs still describe only V2:
    • Intro: add ZipDepth (inverse depth) and DA3 (depth, larger is farther).
    • Options table: the "Depth Anything V2 preset" column doesn't fit ZipDepth ('linear', { alpha: 1 / 255, beta: 0 }).
    • IMAGENET_NORM is used by V3 too.
  3. Demo: V3 outputs depth, and 1 - normalized squashes near-range contrast (the vase merges with the table). Consider inverting before normalizing:
    const values = isDepth ? depth.data.map((d) => (d > 0 ? 1 / d : NaN)) : depth.data;
  4. Demo: use the option's family instead of Object.values(DEPTH_V3_MODELS).includes(selectedModel).

Optional follow-up: a depthKind: 'inverse' | 'depth' field on DepthEstimatorModel would save apps from special-casing V3.

@msluszniak

Copy link
Copy Markdown
Member

Tested both iOS and Android in demo apps on all possible models and worked really cool!

@barhanc

barhanc commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

FIxed both the HF and applied the code suggestion. Also improved the performance of models after profiling (around 10% inference time improvement).

@barhanc
barhanc requested a review from msluszniak October 20, 2026 14:17
@barhanc
barhanc merged commit b098b90 into main Oct 20, 2026
7 checks passed
@barhanc
barhanc deleted the @bh/depth-anything branch October 20, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature PRs that implement a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Depth Anything V2 model

2 participants