diff --git a/docs/docs/advanced/inference-time-scaling.md b/docs/docs/advanced/inference-time-scaling.md index 392c272b9f..20d70e90fe 100644 --- a/docs/docs/advanced/inference-time-scaling.md +++ b/docs/docs/advanced/inference-time-scaling.md @@ -150,6 +150,17 @@ print(f"Attempts: {len(result.sample_generations)}") The `ValidationResult.reason` string is passed to both S1 and S2 as repair guidance — write specific, actionable failure reasons for best results. +Pass `judge_backend` to have an LLM judge validate any requirement that has no +`validation_fn`. `feedback_strategy` sets how much detail the judge returns to S1 as +repair feedback. It has no effect without `judge_backend`, or for requirements that +have their own `validation_fn`: + +| Strategy | Feedback to S1 | +| -------- | -------------- | +| `"simple"` (default) | Pass/fail only; the judge answers yes or no | +| `"first_error"` | The first mistake the judge finds | +| `"all_errors"` | Every mistake the judge finds | + > **Full example:** [`docs/examples/sofai/sofai_graph_coloring.py`](https://github.com/generative-computing/mellea/blob/main/docs/examples/sofai/sofai_graph_coloring.py) ## Budget forcing diff --git a/docs/examples/aLora/101_example.py b/docs/examples/aLora/101_example.py index aab38e3303..9f1b8e516c 100644 --- a/docs/examples/aLora/101_example.py +++ b/docs/examples/aLora/101_example.py @@ -13,6 +13,15 @@ `stembolts_intrinsic.py` in this directory — it has no standalone entry point of its own; `102_example.py` drives it interactively (reads from stdin, so it isn't runnable as an automated example). + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/aLora/101_example.py """ import time diff --git a/docs/examples/aLora/102_example.py b/docs/examples/aLora/102_example.py index d28afdec71..1198f34367 100644 --- a/docs/examples/aLora/102_example.py +++ b/docs/examples/aLora/102_example.py @@ -1,5 +1,26 @@ # pytest: skip, huggingface, e2e # SKIP REASON: Requires user input (blocks on stdin in an infinite loop). +"""Interactive diagnosis loop using a fully custom, non-catalog aLoRA adapter. + +Demonstrates loading and calling the custom `stembolts` adapter (Hugging Face: +`nfulton/stembolts`) through the helpers in `stembolts_intrinsic.py`. Each turn +runs the adapter-backed intrinsic with a fresh `ChatContext` and prints the raw +JSON result (`defective_part` and `diag_likelihood`). The loop never exits on its +own; stop it with Ctrl+C. + +The example uses Granite 3.3 2B because that is the available `stembolts` +adapter variant; Granite 4.0 and 4.1 variants are not currently available. To +train and upload your own variant, see `README.md` in this directory. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/aLora/102_example.py +""" from stembolts_intrinsic import ( async_stembolt_failure_analysis, diff --git a/docs/examples/aLora/README.md b/docs/examples/aLora/README.md index 62e82fa1e7..c04a018fce 100644 --- a/docs/examples/aLora/README.md +++ b/docs/examples/aLora/README.md @@ -1,23 +1,42 @@ # aLoRA Examples -This directory contains examples demonstrating how to tune and use your own (Adaptive) Low-Rank Adapters. +This directory contains examples demonstrating how to tune and use your own (Activated) +Low-Rank Adapters. For the full CLI reference and a conceptual overview, see +[LoRA and aLoRA adapters](../../docs/advanced/lora-and-alora-adapters.md), +[`m alora` CLI reference](../../docs/reference/cli.md), and the +[custom adapter function tutorial](../../docs/tutorials/07-custom-adapter-function.md). +For the built-in catalog adapters, see [`intrinsics/`](../intrinsics/README.md). + +## Examples + +| File | Description | +| :--- | :--- | +| [`101_example.py`](./101_example.py) | Comparing aLoRA-backed requirement validation against LLM-as-a-judge, using the catalog-native `requirement-check` adapter (not a custom one — see `stembolts_intrinsic.py` for loading your own). | +| [`102_example.py`](./102_example.py) | Interactive loop that exercises the custom `stembolts` adapter — skip-marked because it blocks on stdin in an infinite loop. | +| [`stembolts_intrinsic.py`](./stembolts_intrinsic.py) | Helper module for loading and calling the fully custom `stembolts` adapter (no standalone entry point — `102_example.py` consumes it). | +| [`make_training_data.py`](./make_training_data.py) | Converts raw JSONL into `{item, label}` training data, validating labels against a JSON schema (`jsonschema`). | +| [`example_readme_generator.py`](./example_readme_generator.py) | Programmatic README generation for adapters. | +| [`stembolt_failure_dataset.jsonl`](./stembolt_failure_dataset.jsonl) | Training data in the `{item, label}` format that `m alora train` expects. | +| [`stembolt_customer_data.jsonl`](./stembolt_customer_data.jsonl) | Raw data used as input to `make_training_data.py`. | +| [`io.yaml`](./io.yaml) | Output contract for the `stembolts` adapter, passed to `m alora upload --io-yaml`. | +| [`hints.txt`](./hints.txt) | Domain hints passed to `m alora add-readme --hints`. | ## Training Models First, we need to clone Mellea and put ourselves into this directory: ```bash -git clone github.com/generative-computing/mellea; -cd mellea; -uv venv .venv; -source .venv/bin/activate; -uv pip install -e .[all]; -pushd docs/examples/aLora/; +git clone https://github.com/generative-computing/mellea +cd mellea +uv venv .venv +source .venv/bin/activate +uv pip install -e ".[all]" +pushd docs/examples/aLora/ ``` Now let's train a model: -``` +```bash m alora train \ --basemodel ibm-granite/granite-4.1-3b \ --outfile stembolts_model \ @@ -32,22 +51,23 @@ m alora train \ ## Upload Models -If model training succeeds, you will need to upload your model as an adapter function: +If model training succeeds, you will need to upload your model as an adapter function. +Uploading requires a Hugging Face account; run `huggingface-cli login` first. ```bash # WARNING: running this command will upload your model weights to huggingface.co !!! # The model will be private. # replace $HF_USERNAME with your Hugging Face username. m alora upload \ - --intrinsic \ - --name "$HF_USERNAME/stembolts" \ - --io-yaml io.yaml \ + --intrinsic \ + --name "$HF_USERNAME/stembolts" \ + --io-yaml io.yaml \ stembolts_model ``` You can also train and upload the same adapter for multiple model families: -``` +```bash # CHANGE $HF_USERNAME to your username, or set envvar. m alora train \ --basemodel ibm-granite/granite-3.3-2b-instruct \ @@ -55,15 +75,15 @@ m alora train \ --adapter alora \ stembolt_failure_dataset.jsonl && m alora upload \ - --intrinsic \ - --name "$HF_USERNAME/stembolts" \ - --io-yaml io.yaml \ + --intrinsic \ + --name "$HF_USERNAME/stembolts" \ + --io-yaml io.yaml \ stembolts_model_3.3_2b ``` ## Generate a README -After uploading your adapter, you can auto-generate a README for the Hugging Face model repository using `m alora add-readme`. This command uses Mellea to analyze your training dataset and produce documentation with a description, data examples, and integration code: +After uploading your adapter, you can auto-generate a README for the Hugging Face model repository using `m alora add-readme`. This command uses Mellea to analyze your training dataset and produce documentation with a description, data examples, and integration code. It needs a running model backend (the default session backend, so `ollama serve`): ```bash m alora add-readme \ @@ -80,7 +100,7 @@ m alora add-readme \ --name $HF_USERNAME/stembolts \ --io-yaml io.yaml \ --basemodel granite-4.1-3b \ - --hints hints.txt + --hints hints.txt \ stembolt_failure_dataset.jsonl ``` @@ -88,87 +108,10 @@ The generator will display the README and ask for confirmation before uploading ## Using Adapter Functions -Compose the adapter directly from an `Identity`, an output contract, and a -weights binding — no dedicated adapter class needed: - -```python -from mellea.backends.adapters import Adapter, Identity, LocalFileBinding, get_io_contract -from mellea.backends.adapters.catalog import AdapterType - -def stembolt_adapter(base_model_name: str = "granite-4.1-3b") -> Adapter: - return Adapter( - identity=Identity(name="stembolts", adapter_type="alora"), - # get_io_contract falls back to a permissive dict contract for names - # outside the built-in catalog. - io_contract=get_io_contract("stembolts"), - weights=LocalFileBinding( - name="stembolts", - adapter_type=AdapterType.ALORA, - repo_id="$USERNAME/stembolts", # REPLACE $USERNAME WITH YOUR HUGGING FACE USERNAME - # A custom, non-catalog adapter has no catalog entry to resolve - # revision=None from — pin it explicitly. - revision="main", - ), - ) -``` - -Registering it with a backend downloads and loads the weights: - -```python -from mellea.backends.cache import SimpleLRUCache -from mellea.backends.huggingface import LocalHFBackend - -backend = LocalHFBackend( - model_id="ibm-granite/granite-4.1-3b", cache=SimpleLRUCache(5) -) - -backend.add_adapter(stembolt_adapter(base_model_name="granite-4.1-3b")) -``` - -## Example Files - -### [101_example.py](./101_example.py) -Comparing aLoRA-backed requirement validation against LLM-as-a-judge, using the -catalog-native `requirement-check` adapter (not a custom one — see -`stembolts_intrinsic.py` for loading your own). - -Demonstrates: -- Registering a catalog adapter as an aLoRA -- `ALoraRequirement` vs `LLMaJRequirement` routing -- Timing both validation paths - -### [102_example.py](./102_example.py) -Interactive loop that exercises the custom `stembolts` adapter — skip-marked -because it blocks on stdin in an infinite loop. - -The example uses Granite 3.3 2B because that is the available `stembolts` -adapter variant; Granite 4.0 and 4.1 variants are not currently available. - -Demonstrates: -- Loading a fully custom, non-catalog adapter via `stembolts_intrinsic.py` -- Running the adapter-backed intrinsic repeatedly with a fresh `ChatContext` - -### [make_training_data.py](./make_training_data.py) -Utility for preparing training datasets for adapter training. - -Demonstrates: -- Dataset format preparation -- Data validation -- Preprocessing for training - -### [stembolts_intrinsic.py](./stembolts_intrinsic.py) -Helper module for loading and calling the fully custom `stembolts` adapter -(no standalone entry point — `102_example.py` consumes it). - -Demonstrates: -- Custom adapter composition (`_stembolt_adapter()`) and intrinsic definition (`StemboltIntrinsic`) -- Registering the adapter on a backend, guarded by qualified name -- Calling the intrinsic directly via `mfuncs.act` - -### [example_readme_generator.py](./example_readme_generator.py) -Programmatic README generation for adapters. - -Demonstrates: -- Auto-generating adapter documentation -- Using the `m alora add-readme` command programmatically -- Documenting custom adapters +Compose the adapter from an `Identity`, an output contract, and a `LocalFileBinding`, +then register it on a `LocalHFBackend` with `backend.add_adapter()`. The code is in +[LoRA and aLoRA adapters](../../docs/advanced/lora-and-alora-adapters.md), and +`stembolts_intrinsic.py` is the helper module `102_example.py` imports to load the +`stembolts` adapter; replace its `repo_id` (`nfulton/stembolts`) with `$HF_USERNAME/stembolts` to use your +own upload. A custom, non-catalog adapter has no catalog entry to resolve `revision` +from, so pin it explicitly (`revision="main"`). diff --git a/docs/examples/aLora/example_readme_generator.py b/docs/examples/aLora/example_readme_generator.py index c7369ed34e..c93a5755f2 100644 --- a/docs/examples/aLora/example_readme_generator.py +++ b/docs/examples/aLora/example_readme_generator.py @@ -1,5 +1,21 @@ # pytest: skip, huggingface, e2e # SKIP REASON: documentation only. +"""Generating a Hugging Face README for a trained adapter programmatically. + +Demonstrates calling `generate_readme()` directly, which is the same code path as +the `m alora add-readme` command. A `MelleaSession` analyzes the training dataset +and writes a model README (description, data examples, integration code) to +`stembolts_model_readme.md`. Replace the placeholder `name` with your own +`/`; `hints` and `prompt_file` are optional. + +Prerequisites: + uv sync --extra cli + ollama serve + ollama pull granite4.2:3b + +Usage: + cd docs/examples/aLora && uv run python example_readme_generator.py +""" from cli.alora.readme_generator import generate_readme, make_readme_jinja_dict diff --git a/docs/examples/aLora/make_training_data.py b/docs/examples/aLora/make_training_data.py index b9b26b1d0f..fed5e4caaa 100644 --- a/docs/examples/aLora/make_training_data.py +++ b/docs/examples/aLora/make_training_data.py @@ -1,5 +1,18 @@ # pytest: skip, huggingface, e2e # SKIP REASON: documentation only. +r"""Converting raw JSONL into the `{item, label}` training format for aLoRA training. + +Reads a JSONL file whose rows carry `item`, `defective_part`, and `diag_likelihood` +(such as `stembolt_customer_data.jsonl`), validates each label against the +stembolt response schema, and writes rows of the form +`{"item": ..., "label": ""}` — the format `m alora train` expects (see +`stembolt_failure_dataset.jsonl`). Output goes to stdout unless `-o` is given. + +Usage: + uv run python docs/examples/aLora/make_training_data.py \ + docs/examples/aLora/stembolt_customer_data.jsonl -o training_data.jsonl +""" + import argparse import json import sys diff --git a/docs/examples/aLora/stembolts_intrinsic.py b/docs/examples/aLora/stembolts_intrinsic.py index 26f0bf86e4..a673ce98a4 100644 --- a/docs/examples/aLora/stembolts_intrinsic.py +++ b/docs/examples/aLora/stembolts_intrinsic.py @@ -8,6 +8,15 @@ most catalog adapters use. It is invoked directly via `Intrinsic`/`mfuncs.act()` and the raw JSON is returned to the caller — it is not a pass/fail `ALoraRequirement` validator. Consumed by `102_example.py` in this directory. + +This module has no standalone entry point and is not run directly. + +Prerequisites: + uv sync --extra hf + +Usage: + See `102_example.py`, which imports `stembolt_failure_analysis()` and + `async_stembolt_failure_analysis()` from this module. """ import mellea.stdlib.functional as mfuncs diff --git a/docs/examples/granite-switch/README.md b/docs/examples/granite-switch/README.md index d916549de9..3d8ddcee5f 100644 --- a/docs/examples/granite-switch/README.md +++ b/docs/examples/granite-switch/README.md @@ -2,9 +2,11 @@ This directory contains examples for running Mellea adapter functions with Granite Switch models, either locally through `LocalHFBackend` or through an -OpenAI-compatible vLLM deployment. - -## What is Granite Switch? +OpenAI-compatible vLLM deployment. For setup and backend details, see +[Adapter functions](../../docs/advanced/intrinsics.md) and the +[official Granite Switch documentation](https://github.com/generative-computing/granite-switch). +For runtime LoRA/aLoRA adapters on a standard backend, see +[`intrinsics/`](../intrinsics/README.md). Granite Switch models ship with LoRA and aLoRA adapters pre-baked into the model weights. Unlike runtime LoRA/aLoRA adapters on a standard `LocalHFBackend`, @@ -12,62 +14,19 @@ these embedded adapters are activated via control tokens injected by the model's chat template. Only the I/O transformation configs are downloaded — no adapter weights are transferred. -## Prerequisites - -### Local Hugging Face inference - -Run the local example on a GPU or Apple Silicon Mac: - -```bash -uv sync --extra hf -``` - -### OpenAI-compatible inference - -1. Host a Granite Switch model with [vLLM](https://docs.vllm.ai/). -2. Install the `switch` extra to download embedded adapter metadata: - -```bash -uv sync --extra switch -``` - -## Available adapters - Not all adapter functions are embedded in every Granite Switch model. Check the model's `adapter_index.json` for a definitive list. For Granite Switch models pre-built by IBM, Mellea includes a list of models in `model_id`. -## Files +## Examples -### [answerability_local_hf.py](./answerability_local_hf.py) - -Demonstrates `rag.check_answerability()` against a local Granite Switch -checkpoint using `LocalHFBackend(load_embedded_adapters=True)`. - -### [answerability_openai.py](./answerability_openai.py) - -Demonstrates `rag.check_answerability()` using `OpenAIBackend` with -`load_embedded_adapters=True` — the simplest way to use adapter functions with Granite -Switch. - -### [hallucination_detection_openai.py](./hallucination_detection_openai.py) - -Demonstrates `rag.flag_hallucinated_content()` using `OpenAIBackend` with -`load_embedded_adapters=True`. - -### [manual_adapter_loading.py](./manual_adapter_loading.py) - -Shows how to manually load embedded adapters using -`backend.register_embedded_adapter_model(intrinsic_name=...)`. Useful when -you only need a subset of adapters or want more control over adapter -registration. +| File | Description | +| :--- | :--- | +| [`answerability_local_hf.py`](./answerability_local_hf.py) | Demonstrates `rag.check_answerability()` against a local Granite Switch checkpoint using `LocalHFBackend(load_embedded_adapters=True)`. Needs a GPU or Apple Silicon Mac (`uv sync --extra hf`). | +| [`answerability_openai.py`](./answerability_openai.py) | Demonstrates `rag.check_answerability()` using `OpenAIBackend` with `load_embedded_adapters=True` — the simplest way to use adapter functions with Granite Switch. Needs a running vLLM server (`uv sync --extra switch`). | +| [`hallucination_detection_openai.py`](./hallucination_detection_openai.py) | Demonstrates `rag.flag_hallucinated_content()` using `OpenAIBackend` with `load_embedded_adapters=True`. Needs a running vLLM server (`uv sync --extra switch`). | +| [`manual_adapter_loading.py`](./manual_adapter_loading.py) | Shows how to manually load embedded adapters using `backend.register_embedded_adapter_model(intrinsic_name=...)`. Useful when you only need a subset of adapters or want more control over adapter registration. Needs a running vLLM server (`uv sync --extra switch`). | ## Architecture -![Granite Libraries Software Stack Architecture in Mellea](../../docs/images/granite-libraries-mellea-architecture.png) - -## Related - -- [`../intrinsics/`](../intrinsics/) — runtime LoRA/aLoRA adapter functions -- [Adapter Functions Documentation](../../docs/advanced/intrinsics.md) -- [Official Granite Switch Documentation](https://github.com/generative-computing/granite-switch) +![Granite Libraries Software Stack Architecture in Mellea](../../static/images/granite-libraries-mellea-architecture.png) diff --git a/docs/examples/granite-switch/answerability_local_hf.py b/docs/examples/granite-switch/answerability_local_hf.py index cfd637a149..03a161c5aa 100644 --- a/docs/examples/granite-switch/answerability_local_hf.py +++ b/docs/examples/granite-switch/answerability_local_hf.py @@ -1,13 +1,17 @@ # pytest: e2e, huggingface, skip - """Run the answerability adapter function through a local Granite Switch checkpoint. -Requires a GPU or Apple Silicon Mac and: +Demonstrates `rag.check_answerability()` against a Granite Switch checkpoint loaded +with `LocalHFBackend(load_embedded_adapters=True)`. The adapters are already baked +into the checkpoint, so no separate adapter weights are downloaded. - uv sync --extra hf +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the checkpoint is +downloaded from Hugging Face on first run. -To run from the Mellea source tree: +Prerequisites: + uv sync --extra hf +Usage: uv run python docs/examples/granite-switch/answerability_local_hf.py """ diff --git a/docs/examples/granite-switch/answerability_openai.py b/docs/examples/granite-switch/answerability_openai.py index 4d9fedf573..1c5b2091ce 100644 --- a/docs/examples/granite-switch/answerability_openai.py +++ b/docs/examples/granite-switch/answerability_openai.py @@ -1,15 +1,20 @@ # pytest: e2e, vllm, skip +r"""Example: running the answerability adapter function via OpenAI backend with Granite Switch. -"""Example: running the answerability adapter function via OpenAI backend with Granite Switch. +Demonstrates `rag.check_answerability()` through `OpenAIBackend` with +`load_embedded_adapters=True` — the simplest way to use adapter functions with +Granite Switch. The check runs twice: once with a document that contains the +answer and once with a document that does not. The script exits with status 1 and a +"Skipped" message if the vLLM server is not reachable (set `VLLM_SWITCH_TEST_BASE_URL` +to override the default `http://localhost:8000`). -Requires a vLLM server hosting a Granite Switch model. - -To start the server: +Prerequisites: + uv sync --extra switch # downloads the embedded adapter metadata python -m vllm.entrypoints.openai.api_server \ --model \ --dtype bfloat16 --enable-prefix-caching -To run this script from the root of the Mellea source tree: +Usage: uv run python docs/examples/granite-switch/answerability_openai.py """ diff --git a/docs/examples/granite-switch/hallucination_detection_openai.py b/docs/examples/granite-switch/hallucination_detection_openai.py index faaf954216..c6bf96b178 100644 --- a/docs/examples/granite-switch/hallucination_detection_openai.py +++ b/docs/examples/granite-switch/hallucination_detection_openai.py @@ -1,15 +1,18 @@ # pytest: e2e, vllm, skip +r"""Example: running hallucination detection via OpenAI backend with Granite Switch. -"""Example: running hallucination detection via OpenAI backend with Granite Switch. +Demonstrates `rag.flag_hallucinated_content()` through `OpenAIBackend` with +`load_embedded_adapters=True`. The script exits with status 1 and a "Skipped" +message if the vLLM server is not reachable (set `VLLM_SWITCH_TEST_BASE_URL` to +override the default `http://localhost:8000`). -Requires a vLLM server hosting a Granite Switch model. - -To start the server: +Prerequisites: + uv sync --extra switch # downloads the embedded adapter metadata python -m vllm.entrypoints.openai.api_server \ --model \ --dtype bfloat16 --enable-prefix-caching -To run this script from the root of the Mellea source tree: +Usage: uv run python docs/examples/granite-switch/hallucination_detection_openai.py """ diff --git a/docs/examples/granite-switch/manual_adapter_loading.py b/docs/examples/granite-switch/manual_adapter_loading.py index 13c5a1bc0c..ec1243931f 100644 --- a/docs/examples/granite-switch/manual_adapter_loading.py +++ b/docs/examples/granite-switch/manual_adapter_loading.py @@ -1,6 +1,5 @@ # pytest: e2e, vllm, skip - -"""Example: manually loading embedded adapters for the OpenAI backend. +r"""Example: manually loading embedded adapters for the OpenAI backend. Instead of using `load_embedded_adapters=True` (which loads all adapters from the model repo at init), this example shows how to create an OpenAIBackend with @@ -14,14 +13,17 @@ - You want to load adapters from a local directory rather than Hugging Face Hub. - You want more control over adapter registration. -Requires a vLLM server hosting a Granite Switch model. +The script exits with status 1 and a "Skipped" message if the vLLM server is not +reachable (set `VLLM_SWITCH_TEST_BASE_URL` to override the default +`http://localhost:8000`). -To start the server: +Prerequisites: + uv sync --extra switch # downloads the embedded adapter metadata python -m vllm.entrypoints.openai.api_server \ --model \ --dtype bfloat16 --enable-prefix-caching -To run this script from the root of the Mellea source tree: +Usage: uv run python docs/examples/granite-switch/manual_adapter_loading.py """ diff --git a/docs/examples/intrinsics/README.md b/docs/examples/intrinsics/README.md index 4c3c660c6e..abc811df9e 100644 --- a/docs/examples/intrinsics/README.md +++ b/docs/examples/intrinsics/README.md @@ -1,169 +1,32 @@ # Adapter Functions Examples -This directory contains examples for using Mellea's adapter functions - specialized model capabilities accessed through adapters. - -## Concepts Demonstrated - -- **Adapter Functions**: Specialized model capabilities beyond text generation -- **Adapter System**: Using LoRA/aLoRA adapters for specific tasks -- **RAG Evaluation**: Assessing retrieval-augmented generation quality -- **Quality Metrics**: Measuring relevance, groundedness, and accuracy -- **Backend Integration**: Running adapter functions through LocalHFBackend - (runtime adapters or local Granite Switch checkpoints) and OpenAIBackend - (Granite Switch deployments) - -## Basic Usage - -```python -from mellea import model_ids, start_backend -from mellea.stdlib import functional as mfuncs -from mellea.stdlib.components.intrinsic import core - -ctx, backend = start_backend( - "hf", model_id=model_ids.IBM_GRANITE_4_1_3B, context_type="chat" -) - -response, ctx = mfuncs.chat("What is 2 + 2?", ctx, backend) -print(f"Response: {response.content}") - -# NOTE: There are additional functions for other adapter functions as well. -result = core.check_certainty(ctx, backend) -print(f"Certainty score: {result}") -``` - -Granite Switch models can run locally with `LocalHFBackend`: - -```python -from mellea.backends.huggingface import LocalHFBackend -from mellea.backends.model_ids import IBM_GRANITE_SWITCH_4_1_3B_PREVIEW - -backend = LocalHFBackend( - model_id=IBM_GRANITE_SWITCH_4_1_3B_PREVIEW, - load_embedded_adapters=True, -) -``` - -Install this path with `uv sync --extra hf`. - -The underlying adapter functions can also be utilized directly when generating: - -```python -from mellea.stdlib.components import Intrinsic -import mellea.stdlib.functional as mfuncs - -... - -out, new_ctx = mfuncs.act( - Intrinsic( - "requirement-check", - intrinsic_kwargs={"requirement": "The assistant is helpful."}), - ctx, - backend -) -``` - -For complete runnable Granite Switch examples, see -[`../granite-switch/`](../granite-switch/). - -> **Note:** Not all adapter functions are embedded in every Granite Switch model. You should check -> the model's `adapter_index.json` file for a definitive list. For granite switch models -> pre-built by IBM, we include a list of models in the Mellea `model_id`. - -## Available Adapter Functions - -- **answerability**: Determine if question is answerable -- **citations**: Extract and validate citations -- **context-attribution**: Identify context sentences that most influenced response -- **factuality_correction**: Correct factually incorrect responses -- **factuality_detection**: Detect factually incorrect responses -- **guardian-core**: Safety risk detection (harm, bias, groundedness, custom criteria) -- **hallucination_detection**: Detect hallucinated content -- **policy_guardrails**: Determine if scenario complies with policy -- **query_clarification**: Generate a clarification request if needed, otherwise "CLEAR". -- **query_rewrite**: Improve query formulation -- **requirement_check**: Validate requirements (used by ALoraRequirement) -- **uncertainty**: Estimate certainty about answering a question - -## Example Files - -### RAG Adapter Functions - -```bash -uv run answerability.py -``` -Demonstrates checking if documents can answer a question. - -```bash -uv run citations.py -``` -Demonstrates extracting and validating citations from responses. - -```bash -uv run context_attribution.py -``` -Demonstrates identifying which context sentences influenced the response. - -```bash -uv run hallucination_detection.py -``` -Demonstrates detecting potentially hallucinated content. - -```bash -uv run query_clarification.py -``` -Demonstrates generating clarification requests when needed. - -```bash -uv run query_rewrite.py -``` -Demonstrates rewriting queries for better retrieval. - -### Core Adapter Functions - -```bash -uv run requirement_check.py -``` -Demonstrates validating requirements against model outputs. - -```bash -uv run uncertainty.py -``` -Demonstrates assessing model certainty about its response. - -### Guardian Adapter Functions - -```bash -uv run factuality_detection.py -``` -Demonstrates detecting factually incorrect statements. - -```bash -uv run factuality_correction.py -``` -Demonstrates automatically correcting factual errors. - -```bash -uv run guardian_core.py -``` -Demonstrates comprehensive safety checks (harm, bias, groundedness). - -```bash -uv run policy_guardrails.py -``` -Demonstrates checking compliance with policies. - -### Comprehensive Example - -```bash -uv run intrinsics.py -``` -Full example showing multiple adapter functions working together in a RAG pipeline. +This directory contains examples for using Mellea's adapter functions - specialized +model capabilities accessed through adapters. For setup, backend support, and a +conceptual overview, see [Adapter functions](../../docs/advanced/intrinsics.md) and +[Safety guardrails](../../docs/how-to/safety-guardrails.md). Not all adapter +functions are embedded in every Granite Switch model; for Granite Switch examples, +see [`granite-switch/`](../granite-switch/README.md). To train your own adapter, see +[`aLora/`](../aLora/README.md). + +## Examples + +| File | Description | +| :--- | :--- | +| [`answerability.py`](./answerability.py) | Demonstrates checking if documents can answer a question. | +| [`citations.py`](./citations.py) | Demonstrates extracting and validating citations from responses. | +| [`context_attribution.py`](./context_attribution.py) | Demonstrates identifying which context sentences influenced the response. | +| [`hallucination_detection.py`](./hallucination_detection.py) | Demonstrates detecting potentially hallucinated content. | +| [`query_clarification.py`](./query_clarification.py) | Demonstrates generating clarification requests when needed. | +| [`query_rewrite.py`](./query_rewrite.py) | Demonstrates rewriting queries for better retrieval. | +| [`requirement_check.py`](./requirement_check.py) | Demonstrates validating requirements against model outputs. | +| [`uncertainty.py`](./uncertainty.py) | Demonstrates assessing model certainty about its response. | +| [`uncertainty_ollama.py`](./uncertainty_ollama.py) | `core.check_certainty()` on `OllamaModelBackend` using an Ollama model that bundles the uncertainty aLoRA (deprecated, see [#1633](https://github.com/generative-computing/mellea/issues/1633)). | +| [`factuality_detection.py`](./factuality_detection.py) | Demonstrates detecting factually incorrect statements. | +| [`factuality_correction.py`](./factuality_correction.py) | Demonstrates automatically correcting factual errors. | +| [`guardian_core.py`](./guardian_core.py) | Demonstrates comprehensive safety checks (harm, bias, groundedness). | +| [`policy_guardrails.py`](./policy_guardrails.py) | Demonstrates checking compliance with policies. | +| [`intrinsics.py`](./intrinsics.py) | Direct use of an adapter function via a composed `Adapter`, `Intrinsic`, and `mfuncs.act()`, underneath the helper functions above. | ## Architecture -![Granite Libraries Software Stack Architecture in Mellea](../../docs/images/granite-libraries-mellea-architecture.png) - -## Related Documentation -- See `mellea/stdlib/components/intrinsic/` for adapter function implementations -- See `mellea/backends/adapters/` for adapter system -- See `docs/docs/examples/granite-switch/README.md` for more about granite-switch +![Granite Libraries Software Stack Architecture in Mellea](../../static/images/granite-libraries-mellea-architecture.png) diff --git a/docs/examples/intrinsics/answerability.py b/docs/examples/intrinsics/answerability.py index 25bbc10ec8..34f5baecae 100644 --- a/docs/examples/intrinsics/answerability.py +++ b/docs/examples/intrinsics/answerability.py @@ -1,11 +1,19 @@ # pytest: huggingface, e2e - """Example usage of the answerability intrinsic for RAG applications. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/answerability.py -``` +Demonstrates `rag.check_answerability()`, which scores whether a set of retrieved +documents contains enough information to answer a question. The script runs the +check twice: once with a document that contains the answer and once with a +document that does not. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/answerability.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/citations.py b/docs/examples/intrinsics/citations.py index 0278a07902..76f60aa03c 100644 --- a/docs/examples/intrinsics/citations.py +++ b/docs/examples/intrinsics/citations.py @@ -1,11 +1,18 @@ # pytest: huggingface, e2e - """Example usage of the citations intrinsic for RAG applications. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/citations.py -``` +Demonstrates `rag.find_citations()`, which identifies the document sentences that +support each sentence of an assistant response. One of the two documents is +deliberately unrelated, so the output shows that only relevant sentences are cited. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/citations.py """ import json diff --git a/docs/examples/intrinsics/context_attribution.py b/docs/examples/intrinsics/context_attribution.py index 73ffee19ea..81f399b2f9 100644 --- a/docs/examples/intrinsics/context_attribution.py +++ b/docs/examples/intrinsics/context_attribution.py @@ -1,15 +1,18 @@ # pytest: huggingface, e2e - """Example usage of the context attribution intrinsic. Intrinsic function that finds sentences in prior conversation messages and RAG documents that were most important to the LLM in generating each sentence in the assistant response. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/context_attribution.py -``` +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/context_attribution.py """ import json diff --git a/docs/examples/intrinsics/factuality_correction.py b/docs/examples/intrinsics/factuality_correction.py index f628a7d8df..b02ea9a92a 100644 --- a/docs/examples/intrinsics/factuality_correction.py +++ b/docs/examples/intrinsics/factuality_correction.py @@ -1,11 +1,19 @@ # pytest: huggingface, e2e - """Example usage of the factuality correction intrinsic. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/factuality_correction.py -``` +Demonstrates `guardian.factuality_correction()`, which rewrites an assistant +response so that it agrees with the supporting document attached to the context. +The document can be attached to the context or passed explicitly with +`documents=`. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/factuality_correction.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/factuality_detection.py b/docs/examples/intrinsics/factuality_detection.py index 7aae199878..474bf395c7 100644 --- a/docs/examples/intrinsics/factuality_detection.py +++ b/docs/examples/intrinsics/factuality_detection.py @@ -1,11 +1,19 @@ # pytest: huggingface, e2e - """Example usage of the factuality detection intrinsic. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/factuality_detection.py -``` +Demonstrates `guardian.factuality_detection()`, which returns "yes" if an assistant +response is factually incorrect given the supporting document attached to the +context, and "no" otherwise. The document can be attached to the context or passed +explicitly with `documents=`. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/factuality_detection.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/guardian_core.py b/docs/examples/intrinsics/guardian_core.py index 035cc5f68d..03b1e63500 100644 --- a/docs/examples/intrinsics/guardian_core.py +++ b/docs/examples/intrinsics/guardian_core.py @@ -1,15 +1,22 @@ # pytest: huggingface, e2e - """Example usage of the guardian-core intrinsic for safety and hallucination detection. Demonstrates how to use the Granite Guardian 4.0 Micro LoRA adapter (guardian-core) through Mellea's intrinsic pipeline for prompt-level risk detection, response-level risk detection, and RAG groundedness checking. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/guardian_core.py -``` +The script also covers custom criteria (PII), function-call hallucination, and +answer relevance. Each check returns a risk score; the script treats a score of +0.5 or higher as a detected risk. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/guardian_core.py """ import json diff --git a/docs/examples/intrinsics/hallucination_detection.py b/docs/examples/intrinsics/hallucination_detection.py index 86074251fa..aeebf02e82 100644 --- a/docs/examples/intrinsics/hallucination_detection.py +++ b/docs/examples/intrinsics/hallucination_detection.py @@ -1,11 +1,17 @@ # pytest: huggingface, e2e - """Example usage of the hallucination detection intrinsic for RAG applications. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/hallucination_detection.py -``` +Demonstrates `rag.flag_hallucinated_content()`, which flags the sentences of an +assistant response that are not supported by the retrieved documents. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/hallucination_detection.py """ import json diff --git a/docs/examples/intrinsics/intrinsics.py b/docs/examples/intrinsics/intrinsics.py index 2a499a8014..bc115a05b1 100644 --- a/docs/examples/intrinsics/intrinsics.py +++ b/docs/examples/intrinsics/intrinsics.py @@ -1,4 +1,27 @@ # pytest: huggingface, e2e +"""Using an adapter function directly with Intrinsic and mfuncs.act(). + +Demonstrates the lower-level path underneath the `rag`, `core`, and `guardian` +helper functions: composing an `Adapter` (an `Identity`, an I/O contract, and a +`LocalFileBinding`) for the catalog's `requirement-check` aLoRA, registering it on +a `LocalHFBackend` with `add_adapter()`, and invoking it with +`mfuncs.act(Intrinsic(...))`. The raw JSON output +(`{"requirement_check": {"score": ...}}`) is printed; `ALoraRequirement` parses +the same output when validating. Prefer the helper functions for everyday use; +see `mellea/stdlib/components/intrinsic/` for them. + +A commented-out alternative shows loading a local Granite Switch checkpoint +instead; see `docs/examples/granite-switch/answerability_local_hf.py`. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/intrinsics.py +""" import mellea.stdlib.functional as mfuncs from mellea.backends import model_ids @@ -13,7 +36,7 @@ from mellea.stdlib.components import Intrinsic, Message from mellea.stdlib.context import ChatContext -# This is an example for how you would directly use intrinsics. See `mellea/stdlib/intrinsics/rag.py` +# This is an example for how you would directly use intrinsics. See `mellea/stdlib/components/intrinsic/rag.py` # for helper functions. backend = LocalHFBackend(model_id=model_ids.IBM_GRANITE_4_1_3B) diff --git a/docs/examples/intrinsics/policy_guardrails.py b/docs/examples/intrinsics/policy_guardrails.py index 96624f293b..19d88e781c 100644 --- a/docs/examples/intrinsics/policy_guardrails.py +++ b/docs/examples/intrinsics/policy_guardrails.py @@ -1,14 +1,17 @@ # pytest: huggingface, e2e - """Example usage of the policy_guardrails intrinsic. Demonstrates how to use the Granite Guardian 4.0 Micro LoRA adapter (policy_guardrails) through Mellea's intrinsic pipeline for scenario-policy compliance checking. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/policy_guardrails.py -``` +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/policy_guardrails.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/query_clarification.py b/docs/examples/intrinsics/query_clarification.py index 25e48394aa..2819f89c59 100644 --- a/docs/examples/intrinsics/query_clarification.py +++ b/docs/examples/intrinsics/query_clarification.py @@ -1,11 +1,18 @@ # pytest: huggingface, e2e -""" -Example usage of the query clarification intrinsic for RAG applications. +"""Example usage of the query clarification intrinsic for RAG applications. + +Demonstrates `rag.clarify_query()`, which returns a clarification request when a +query is ambiguous given the retrieved documents, or "CLEAR" when it is not. The +script runs one ambiguous query and one clear query. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/query_clarification.py -``` +Usage: + uv run python docs/examples/intrinsics/query_clarification.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/query_rewrite.py b/docs/examples/intrinsics/query_rewrite.py index 943fd8014b..c1f72d2424 100644 --- a/docs/examples/intrinsics/query_rewrite.py +++ b/docs/examples/intrinsics/query_rewrite.py @@ -1,3 +1,20 @@ +# pytest: huggingface, e2e +"""Rewriting a follow-up question into a standalone retrieval query with the query-rewrite adapter. + +Demonstrates `rag.rewrite_question()`, which uses the conversation history to +rewrite an under-specified last user question (here, "is he more likely to get +fleas?") into a self-contained query suitable for retrieval. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/query_rewrite.py +""" + from mellea import model_ids, start_backend from mellea.stdlib.components import Message from mellea.stdlib.components.intrinsic import rag diff --git a/docs/examples/intrinsics/requirement_check.py b/docs/examples/intrinsics/requirement_check.py index 58da63d9ce..60209abde2 100644 --- a/docs/examples/intrinsics/requirement_check.py +++ b/docs/examples/intrinsics/requirement_check.py @@ -1,13 +1,17 @@ # pytest: huggingface, e2e - """Example usage of the requirement check intrinsic. Intrinsic function that determines if the text satisfies the given requirements. +This is the same adapter function that `ALoraRequirement` uses internally. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/requirement_check.py -``` +Usage: + uv run python docs/examples/intrinsics/requirement_check.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/uncertainty.py b/docs/examples/intrinsics/uncertainty.py index 3982f0c3ae..34b82a5ff5 100644 --- a/docs/examples/intrinsics/uncertainty.py +++ b/docs/examples/intrinsics/uncertainty.py @@ -1,14 +1,19 @@ # pytest: huggingface, e2e - """Example usage of the uncertainty/certainty intrinsic. Evaluates how certain the model is about its response to a user question. The context should contain a user question followed by an assistant answer. -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/uncertainty.py -``` +For Ollama, see `uncertainty_ollama.py`. + +Runs the model locally, so it needs a GPU or Apple Silicon Mac; the model and +adapter are downloaded from Hugging Face on first run. + +Prerequisites: + uv sync --extra hf + +Usage: + uv run python docs/examples/intrinsics/uncertainty.py """ from mellea import model_ids, start_backend diff --git a/docs/examples/intrinsics/uncertainty_ollama.py b/docs/examples/intrinsics/uncertainty_ollama.py index f641aa07b7..1d6a67dcfa 100644 --- a/docs/examples/intrinsics/uncertainty_ollama.py +++ b/docs/examples/intrinsics/uncertainty_ollama.py @@ -1,20 +1,30 @@ # pytest: e2e, ollama - """Example usage of the uncertainty/certainty intrinsic with Ollama. +**Deprecated:** Ollama v0.34.1 and later can no longer build LoRA/aLoRA adapter +models, so the build step below fails on current Ollama and the script exits +without running. It only runs on Ollama v0.34.0 or earlier (CI pins 0.33.1), or with +`MELLEA_OLLAMA_UNCERTAINTY_MODEL` set to a tag built on one. See +https://github.com/generative-computing/mellea/issues/1633. + Evaluates how certain the model is about its response to a user question. The context should contain a user question followed by an assistant answer. Ollama bundles one adapter per model. For this single-adapter example, the uncertainty aLoRA tag is used for normal chat and the certainty helper, so -Ollama can retain one model identity. +Ollama can retain one model identity (mapped through `adapter_models`). Unless +`MELLEA_OLLAMA_UNCERTAINTY_MODEL` names an existing tag, the script builds one by +running `test/scripts/build_ollama_uncertainty_adapter.sh` (needs `git`, `uv`, and +network access to download the base model and adapter). If the build fails, the +script prints "SKIPPED" and exits with status 0, which the examples runner reports +as a pass. -Requires `mellea[switch]` to download the adapter's `io.yaml`. +Prerequisites: + uv sync --extra switch # downloads the adapter's io.yaml + ollama serve -To run this script from the root of the Mellea source tree, use the command: -``` -uv run python docs/examples/intrinsics/uncertainty_ollama.py -``` +Usage: + uv run python docs/examples/intrinsics/uncertainty_ollama.py """ import os diff --git a/docs/examples/sofai/README.md b/docs/examples/sofai/README.md index 5e43ef6b71..5986f63d34 100644 --- a/docs/examples/sofai/README.md +++ b/docs/examples/sofai/README.md @@ -1,26 +1,15 @@ # SOFAI (Slow and Fast AI) Examples -This directory contains examples of the SOFAI sampling strategy - a two-tier approach using fast and slow models. +This directory contains examples of the SOFAI sampling strategy - a two-tier approach +using fast and slow models. For usage guidance, `s2_solver_mode`, `judge_backend`, and +`feedback_strategy`, see +[Inference-time scaling](../../docs/advanced/inference-time-scaling.md#sofai--dual-model-escalation). -## Files +## Examples -### [sofai_graph_coloring.py](./sofai_graph_coloring.py) -Complete example using SOFAI for constraint satisfaction problems (graph coloring). - -**Key Features:** -- Two-tier model architecture (fast S1, slow S2) -- Iterative feedback loop with fast model -- Escalation to slow model when needed -- Custom validation with detailed feedback -- Consumer-grade hardware friendly - -## Concepts Demonstrated - -- **Two-Tier Architecture**: Fast model for iteration, slow model for hard cases -- **Escalation Strategy**: When to switch from fast to slow model -- **Feedback Loops**: Providing validation feedback to guide generation -- **Constraint Satisfaction**: Solving CSP problems with LLMs -- **Cost Optimization**: Using expensive models only when necessary +| File | Description | +| :--- | :--- | +| [`sofai_graph_coloring.py`](./sofai_graph_coloring.py) | Complete example using SOFAI for constraint satisfaction problems (graph coloring), with a `granite4.1:3b` S1 solver, a `granite4:latest` S2 solver, and a custom validator that returns detailed feedback. Consumer-grade hardware friendly. | ## SOFAI Architecture @@ -41,84 +30,3 @@ Success? ──No→ Escalate to S2 (Slow Model) ↓ ↓ Result Result ``` - -## Basic Usage - -```python -from mellea import start_session -from mellea.backends.ollama import OllamaModelBackend -from mellea.stdlib.sampling import SOFAISamplingStrategy -from mellea.stdlib.requirements import req - -# Create fast and slow backends -s1_backend = OllamaModelBackend(model_id="granite4.1:3b") -s2_backend = OllamaModelBackend(model_id="granite4:latest") - -# Create SOFAI strategy -strategy = SOFAISamplingStrategy( - s1_backend=s1_backend, - s2_backend=s2_backend, - s1_loop_budget=3, # Try fast model 3 times - escalate_on_failure=True -) - -# Use with requirements -m = start_session() -result = m.instruct( - "Solve the problem...", - requirements=[req("Must satisfy constraint X")], - strategy=strategy -) -``` - -## Configuration Options - -```python -SOFAISamplingStrategy( - s1_backend=fast_backend, # Fast model for iteration - s2_backend=slow_backend, # Slow model for escalation - s1_loop_budget=3, # Max attempts with fast model - escalate_on_failure=True, # Use slow model if fast fails - feedback_strategy="all_errors" # How to provide feedback -) -``` - -## Feedback Strategies - -- `"simple"`: Basic pass/fail feedback -- `"first_error"`: Feedback on first failing requirement -- `"all_errors"`: Detailed feedback on all failures - -## Use Cases - -- **Constraint Satisfaction**: Graph coloring, scheduling, planning -- **Complex Reasoning**: Multi-step problems requiring iteration -- **Cost Optimization**: Use expensive models only when needed -- **Quality Assurance**: Fast iteration with slow model fallback - -## Model Selection - -### Fast Models (S1) - -- granite4.1:3b -- llama3.2:3b -- mistral:7b - -### Slow Models (S2) - -- granite4:latest -- llama3:70b -- mixtral:8x7b - -## Performance Tips - -1. **Choose appropriate S1 budget**: Balance speed vs. success rate -2. **Use specific requirements**: Clear constraints help both models -3. **Provide good feedback**: Detailed validation helps iteration -4. **Profile your use case**: Measure when escalation is needed - -## Related Documentation - -- See `mellea/stdlib/sampling/sofai.py` for implementation -- See `test/stdlib/sampling/test_sofai_*.py` for more examples -- See `mellea/stdlib/sampling/` for other sampling strategies \ No newline at end of file diff --git a/docs/examples/sofai/sofai_graph_coloring.py b/docs/examples/sofai/sofai_graph_coloring.py index 5821782e11..f15778c7da 100644 --- a/docs/examples/sofai/sofai_graph_coloring.py +++ b/docs/examples/sofai/sofai_graph_coloring.py @@ -1,5 +1,4 @@ # pytest: ollama, qualitative, e2e - """SOFAI Sampling Strategy Example: Graph Coloring Problem. This example demonstrates the SOFAI (Slow and Fast AI) @@ -16,6 +15,14 @@ - Remove the validation_fn parameter from req() - Add judge_backend and feedback_strategy parameters to SOFAISamplingStrategy - feedback_strategy options: "simple", "first_error", "all_errors" + +Prerequisites: + ollama serve + ollama pull granite4.1:3b + ollama pull granite4:latest + +Usage: + uv run python docs/examples/sofai/sofai_graph_coloring.py """ import json