Skip to content

Add support for the locateanything mmproj projector type used by NVIDIA EAGLE-architecture VLMs (e.g. nvidia/LocateAnything-3B, yuuko-eth/ LocateAnything-3B-GGUF) #1241

Description

@mohanga-qcom

Background

NVIDIA's EAGLE architecture VLMs use a custom projector type locateanything in their mmproj GGUF files. This type is not in the supported projector list in
llama.cpp's clip.cpp at the version bundled with GenieX v0.3.17 (commit ae9291e).

The LLM backbone (qwen2, 3.4B) loads and runs correctly. Only the vision encoder (mmproj) is blocked.

Affected models:

  • nvidia/LocateAnything-3B and its GGUF quantizations (yuuko-eth/LocateAnything-3B-GGUF)
  • Any other EAGLE-architecture VLM using locateanything projector

———

Request

  1. Update the bundled llama.cpp to a version that includes locateanything projector support (check upstream llama.cpp clip.cpp for the relevant commit)
  2. Or add locateanything to the projector type registry in the bundled clip.cpp

———

Reference

Tried on IQ8 device HTP v75.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions