Eredu

eredu noun, Basque — “model; pattern.”

Eredu is a Rust runtime for local language models. It loads supported Hugging Face-style SafeTensors repositories and GGUF checkpoints, prepares tokenizers and chat templates, and provides one API for text, multimodal, and realtime speech models.

The eredu crate is the public entry point. Its default backend is built on MLX, with Metal support on Apple silicon and CPU or CUDA support on x86-64 Linux and Windows. The runtime contracts, artifact inspection, planning, scheduling, and generation orchestration remain backend-neutral and can be built without MLX.

Eredu is independent of Apple and is not an official MLX project.

What it provides

• Local model loading from SafeTensors directories and sharded GGUF files.
• Text, image, video, audio, and realtime codec-token inputs for supported architectures.
• Tokenization, chat templates, reasoning events, constrained native tool calls, and streaming generation.
• Greedy and stochastic sampling plus embedded or external speculative decoding.
• Load-time quantization and bounded weight, cache, and prompt-cache residency.
• Automatic execution planning, admission, telemetry, and supported tensor, pipeline, and expert parallel execution.

See model and checkpoint support for the current family, format, modality, and execution matrix.

Quick start

The minimum supported Rust version is 1.89. Add the main crate to your project:

[dependencies]
eredu = "0.4"

The default features include the MLX backend. A backend-neutral build is also available:

[dependencies]
eredu = { version = "0.4", default-features = false }

To run a supported model from this workspace:

cargo run --release -p eredu-cli -- \
  --model /path/to/model \
  "Write a Rust function that adds two integers."

The CLI is also installable with cargo install eredu-cli.

The CLI accepts local model directories, GGUF files, and model identifiers already present in the local Hugging Face cache. It does not download models. See the CLI guide for automatic planning, quantization, residency, speculative generation, and tool calling.

Workspace crates

Most applications should depend only on eredu. The other crates expose portable components or implementation layers for specialized use.

Crate	Role
eredu	Main model loading, generation, chat, multimodal, realtime, and planning API
eredu-cli	Command-line model runner built on Eredu
eredu-core	Backend-neutral contracts, artifact inspection, planning, validation, and portable schemas
eredu-runtime	Backend-neutral execution, scheduling, cache I/O, and weight and cache residency orchestration
eredu-architectures	Model-family configuration, checkpoint schemas, parameter topology, and backend-neutral execution
eredu-text	Backend-neutral tokenization and chat-template utilities
eredu-gguf	Bounded, framework-independent GGUF reading, writing, and tensor conversion
eredu-backend-mlx	Concrete MLX tensor, execution, checkpoint, composition, media, and codec integration
safemlx	Low-level MLX implementation used by Eredu's default backend
safemlx-sys	MLX C bindings and native build integration
eredu-nn	Backend-neutral tensor and neural-operation contracts for shared architectures
eredu-codec	Backend-neutral neural audio codec architectures used by realtime speech models
eredu-evaluation	Backend-neutral evidence, parity, quality metrics, performance summaries, and evaluation drivers

The workspace also contains procedural macros, integration tests, and an iOS example application.

Documentation

• Documentation index
• Model and checkpoint support
• Model loading, quantization, and memory
• Backend architecture
• Parallel execution
• Native tool calling
• Speculative decoding and MTP
• Platform setup
• CLI guide

All published crates are pre-1.0, so applications should expect API changes between minor releases.

License

The workspace crates are available under MIT or Apache-2.0 unless a crate or vendored component states otherwise. See LICENSE-MIT, LICENSE-APACHE, and the notices inside vendored source trees.