Skip to content

moss_image: the Qwen3.5 prompt encoder as a native Metal forward #1459

Open

Description

@BobClawblaw

MOSS-Image's prompt encoder is a Qwen3.5 text model (32 layers: Gated DeltaNet and gated full attention, hidden 4096, bf16 as published in the checkpoint's text_encoder/). The pipeline formats the prompt with the chat template, runs one causal forward and takes the final-norm rows.

This issue tracks that encoder as a native, prompt-only Metal forward inside the moss_image family (#1456, #1457), so that tf-moss-image takes a prompt string instead of rows prepared in torch:

  • the checkpoint's bf16 weights as published;
  • the native tokenizer;
  • the family's own kernels registered with the kernel contract;
  • each layer's rows checked against the publisher's torch path.

No shared code changes. The image decoder stays out of scope.

Activity

  1. BobClawblaw commented on Oct 27, 2026

    @BobClawblaw
    ContributorAuthor

    /atc claim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions