Skip to content

[PPU][MTHREADS] Update low-precision float (fp8/fp4) support via backend capability contract - #1116

Open
zeroherolin wants to merge 6 commits into
flagos-ai:mainfrom
zeroherolin:feature/low-precision-float
Open

[PPU][MTHREADS] Update low-precision float (fp8/fp4) support via backend capability contract#1116
zeroherolin wants to merge 6 commits into
flagos-ai:mainfrom
zeroherolin:feature/low-precision-float

Conversation

@zeroherolin

@zeroherolin zeroherolin commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR puts low-precision float (fp8/fp4) support on the PPU and MThreads backends under an explicit capability contract, and extends it where gaps remained. Each backend now declares the casts and dot combinations it actually supports; the semantic layer rejects undeclared combinations with the backend's diagnostic and warns at compile time whenever a non-native path runs — nothing degrades silently anymore. Pre-existing paths (PPU's FP16-promotion fp8 dot, MThreads' SQMMA/WMMA fp8 lowering with FMA fallback, fp8 casts, fp4 upcast) keep their behavior, now declared and fixed where broken; newly added are fp8e4nv on PPU cap80–88 via a bit-exact software cast, a working FP4→fp16 upcast below cap89, a rewritten interpreter fp-conversion baseline, and runnable fp8/fp4 tutorials.

Changes

  • Common
    • new: DotCap/DotSupport (NATIVE / NON_NATIVE / UNSUPPORTED) with optional resolve_dot / resolve_dot_scaled codegen queries — UNSUPPORTED combinations raise with the backend's diagnostic, NON_NATIVE ones emit a compile-time warning (native=false).
    • new: capability declaration fields supported_fp8_cast_dtypes / custom_cast_fp8_dtypes (cast whitelist and software-cast routing) and async_copy_dtypes / descriptor_dtypes (data movement; declaration-only at this stage, shipped with h.metadata).
    • new: runnable tutorials python/tutorials/precision/01-fp8.py / 02-fp4.py covering capability queries, saturating casts, fp8 matmul, FP4 decode and dot_scaled.
    • unified: fp8 cast routing generalized from the hardcoded fp8e4b15-only special case to the declared whitelist.
    • fixed: interpreter float conversion rewritten with strict RTNE/RTZ and per-format fp8 special-value handling (NaN codes, fnuz, no-inf saturation).
  • PPU
    • new: fp8e4nv opened on cap80–88 through a bit-exact software cast (frontend convert_custom_float8_sub89 for downcasts, inline-asm upcast core in the backend); native cvt from cap89 unchanged. The type was previously rejected outright below cap89.
    • fixed: FP4→fp16 upcast gated on capability — it unconditionally emitted cap89-only e4m3 cvt instructions; below cap89 it now decodes via the bf16 table with an exact integer rebias.
    • unified: the existing FP16-promotion fp8 dot is declared NON_NATIVE below cap89 and warns; resolve_dot_scaled declares e2m1 native on cap89 and NON_NATIVE decompose otherwise.
  • MThreads
    • unified: fp8 storage/cast dual whitelist regularized (custom_fp8_dtypescustom_cast_fp8_dtypes, cast whitelist derived from storage); the existing SQMMA/WMMA fp8 lowering is declared native for same-type fp8 with an fp32 accumulator on instruction-tile shapes from cap31, the existing FMA fallback is declared NON_NATIVE with a warning; dot_scaled always declares NON_NATIVE decompose; enable_fp8_burst2 moved from a C++-read env var into options; spec overlay kept in sync.
    • fixed: SQMMA operand retyping for fp8 staged through same-width bit containers; WMMA fp16-accumulator bail-out to the FMA fallback; WMMA same-element-type verifier.
  • Tests: PPU fp8 compile-error tests updated for cap80 semantics (fp8e4nv compiles and warns non-native); exhaustive software-cast checks, including strict-RTNE downcast and clamping, enabled on cap80–88.

Compatibility

  • The contract is optional: backends that do not register it keep the legacy dot/cast rules, and defaults preserve existing behavior for all other backends.
  • Interpreter-mode numerics change for every backend as a correctness fix: strict RTNE (previously round-half-up) and per-format fp8 special-value handling.

Supersedes #1112 / #1113 / #1114.

…, fp8 cast dtype routing, interpreter float conversion rewrite, precision tutorials

Common-layer groundwork for per-backend low-precision float support:

- backends/compiler.py: DotCap/DotSupport, the answer type for the new
  optional resolve_dot / resolve_dot_scaled codegen-function queries.
  Backends that do not register them keep the legacy dot dtype rules.
- language/semantic.py: query resolve_dot(_scaled) once shapes are known;
  reject UNSUPPORTED combinations and warn on EMULATED ones. fp8 cast
  routing via supported_fp8_cast_dtypes / custom_cast_fp8_dtypes options;
  the default preserves the legacy fp8e4b15-only custom-cast routing.
- runtime/interpreter.py: rewrite _convert_float as exact decode/encode
  through fp64 with strict RTNE/RTZ and correct fn/fnuz/none special-value
  conventions (the old path rounded half-up and mishandled specials);
  fnuz zero results encode as +0, never the sign-only NaN code. Also add
  the flagtree_hints parameter to create_load/create_masked_load to match
  the semantic layer's call signature.
- backends/__init__.py: dunder probes on the lazy language-extensions
  proxy raise AttributeError instead of failing module resolution.
- tutorials/precision: runnable fp8 (E4M3FN) and fp4 (E2M1) walkthroughs
  driven by the declared capabilities, portable across backends.
…lve_dot capability contract, Fp4ToFp cap gating
… contract, fp8 dual-whitelist field family, SQMMA/WMMA fp8 lowering fixes, enable_fp8_burst2 plumbing
@zeroherolin
zeroherolin force-pushed the feature/low-precision-float branch from 1dc72db to 9a6a282 Compare September 7, 2026 07:32
@zeroherolin zeroherolin changed the title [PPU][MTHREADS] Low-precision float (fp8/fp4) support via backend capability contract [PPU][MTHREADS] Update low-precision float (fp8/fp4) support via backend capability contract Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant