Add @azphalt/llm-host: install and run llm packages in a GitHub sandbox

[HereLiesAz]

This is the host side of kind: "llm". The store can list llm packages, but until now nothing could install or run one.

What it does

• llmConsent(manifest): everything spec/llm.md says a host must show before install:
  • where prompts go and the operator's data handling;
  • the model licence and weights size;
  • the setup token's permissions;
  • which inputs become secrets;
  • every URL setup downloads;
  • the runner's permissions.
• installLlm, which:
  • verifies the .azp first;
  • refuses a public sandbox (it can create one, private, if asked);
  • commits the payload to llm/<id>/ and the runner workflow to .github/workflows/azphalt-llm-<id>.yml in one commit, so several packages share one sandbox;
  • stores inputs as Actions secrets sealed with libsodium crypto_box_seal;
  • runs the one-time op: "setup" task.
• runLlm: the github-actions-runner protocol:
  • dispatches the task, retrying while a just-committed workflow registers;
  • finds the run by run-name, and follows the check run's <seq>\t<message> lines, deduplicated by seq;
  • reads azphalt-llm-result within the reference bounds (4 MB zipped, 2 MB JSON) and inflates only result.json;
  • validates every field;
  • rejects output that contains a session tag;
  • with resume and the same correlation id, follows an already-dispatched run after a restart.
• chatLlm: the openai-chat protocol, with the host translating the delimiters.
• Rolling delimiters: newSessionKey, turnTag, sessionTags, wrap, scrub, translate and containsSessionTag. They're byte-compatible with scripts/llm-sandbox/run.py; the tests pin vectors computed from the Python implementation.

New dependencies: tweetnacl (public domain) and blakejs (MIT), which together make the sealed box GitHub requires for secrets. fflate is already used in the workspace.

Docs: package readme, root README table, ARCHITECTURE.md, and the spec/llm.md status line (plus its docs/specs mirror). Changeset included.

Verification

• 20/20 vitest tests pass. They cover the tag vectors, scrubbing and translation, and a sealed-box round trip decrypted with nacl.box.open. Consent is built from the real Kilo and Qwen registry packages. Install, run and chat are exercised against a fake GitHub: one commit, the executable bit on setup.sh, a sealed secret that decrypts to the input, public-sandbox refusal, tamper refusal, and required inputs. Run covers progress dedupe, tag rejection, a missing artifact, and parseResult bounds and types.
• Typecheck and build are clean, and the built ESM imports and runs under Node.
• Not run: a live install against real GitHub (it needs a user token). The runner side was already exercised for real in Add kind: "llm" and publish first-party llm packages #232.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QwYWVPse8spRrfMMcuTJPv

---

Generated by Claude Code

Summary by Sourcery

Add the host-side implementation for installing and running signed kind: "llm" packages in private GitHub Actions sandboxes.

New Features:

• Add the @azphalt/llm-host package for consent disclosure, private GitHub Actions sandbox installation, model task execution, and direct OpenAI-compatible chat.
• Support rolling session delimiters that sanitize untrusted content and reject leaked session tags across host and runner protocols.
• Provide sealed-box encryption for storing LLM inputs as GitHub Actions secrets.

Enhancements:

• Add bounded, validated artifact result handling, progress tracking, retryable dispatch, and resumable workflow runs for LLM tasks.

Build:

• Add the new package, its dependencies, build configuration, and workspace lockfile entries.

Documentation:

• Document the new package in the root README, architecture documentation, package README, and LLM specification status.

Tests:

• Add coverage for delimiter compatibility, consent generation, encrypted secrets, sandbox installation, chat translation, workflow progress, artifact handling, and output validation.

Chores:

• Add a changeset for the new package.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
azphalt	Error		Sep 27, 2026 10:20pm UTC

[sourcery-ai]

Reviewer's Guide

Introduces @azphalt/llm-host, a host-side implementation of kind:"llm" that discloses installation consent, verifies and installs packages into shared private GitHub Actions sandboxes with sealed secrets, runs and resumes bounded tasks, supports direct OpenAI-compatible chat, and applies reference-compatible rolling delimiters.

Sequence diagram for LLM package installation and setup

sequenceDiagram
    participant Host
    participant GitHub
    participant Sandbox
    participant Runner

    Host->>Host: llmConsent(manifest)
    Host->>Host: verifyAzp(azp)
    Host->>GitHub: ensure private repository
    Host->>GitHub: commitFiles(payload, workflow)
    Host->>GitHub: GET actions secrets public key
    Host->>GitHub: sealedBox(input, public key)
    Host->>GitHub: PUT Actions secret
    Host->>GitHub: dispatch setup task
    GitHub->>Sandbox: start workflow
    Sandbox->>Runner: execute op setup
    Runner-->>GitHub: result artifact and progress
    GitHub-->>Host: setup result

Loading

Sequence diagram for bounded LLM task execution

sequenceDiagram
    participant Host
    participant GitHub
    participant Runner
    participant Artifact

    Host->>GitHub: dispatch workflow task
    loop until run completes
        Host->>GitHub: findRun(install, correlationId)
        Host->>GitHub: read check-run progress
        GitHub-->>Host: seq and message
    end
    Host->>GitHub: list run artifacts
    GitHub-->>Host: azphalt-llm-result
    Host->>Artifact: ghBytes(zip, MAX_ARTIFACT_BYTES)
    Artifact-->>Host: result.json
    Host->>Host: parseResult(raw)
    Host->>Host: containsSessionTag(result, tags)
    Host-->>Host: completed or failed result

Loading

Sequence diagram for direct openai-chat translation

sequenceDiagram
    participant Host
    participant ModelEndpoint

    Host->>Host: sessionTags(sessionKey, turn)
    Host->>Host: translate(messages, tags)
    Host->>ModelEndpoint: POST /chat/completions
    ModelEndpoint-->>Host: chat response
    Host->>Host: containsSessionTag(text, tags)
    Host-->>Host: ChatResult or rejection

Loading

File-Level Changes

Change	Details	Files
Adds a new host package implementing consent disclosure for LLM manifests.	Extracts endpoint, data-handling, licensing, weights, setup permissions, inputs, downloads, and runner permissions into a host-facing consent object. Validates that the manifest is a supported kind:"llm" package.	packages/llm-host/src/consent.ts
Implements installation of verified LLM packages into private GitHub Actions sandboxes.	Verifies the .azp before repository changes and validates sandbox, workflow, and required-input constraints. Creates or checks a private repository and commits package payload plus workflow files atomically, preserving executable shell-script modes. Encrypts configured inputs with GitHub-compatible libsodium sealed boxes and stores them as Actions secrets. Runs the one-time setup task through the runner protocol.	packages/llm-host/src/install.ts packages/llm-host/src/sealed-box.ts packages/llm-host/test/protocol.test.ts packages/llm-host/test/sealed-box.test.ts
Implements the GitHub Actions runner protocol for dispatching tasks and retrieving bounded results.	Retries workflow dispatch while a newly committed workflow registers and supports resuming by correlation ID. Polls workflow runs and check output, deduplicating sequenced progress messages. Downloads and bounds the result artifact, inflates only an appropriately sized result.json, and validates result fields and types. Rejects model output containing rolling session tags.	packages/llm-host/src/run.ts packages/llm-host/src/github.ts packages/llm-host/test/protocol.test.ts
Adds rolling delimiter utilities compatible with the reference Python runner.	Derives per-turn HMAC tags, wraps and scrubs untrusted content, translates tagged segments into user messages, and detects leaked tags. Pins compatibility vectors and tests marker scrubbing, translation, and malformed segments.	packages/llm-host/src/delimiters.ts packages/llm-host/test/delimiters.test.ts
Adds the direct openai-chat host protocol.	Validates endpoint capabilities and authentication requirements, translates rolling delimiters, calls the chat-completions endpoint, and returns text and token usage. Rejects responses containing session tags.	packages/llm-host/src/chat.ts packages/llm-host/test/protocol.test.ts
Packages and documents the new public host library.	Adds package metadata, TypeScript build configuration, public exports, README usage/API documentation, and dependencies. Updates workspace lockfile, changeset, root package table, architecture documentation, and LLM specification status.	packages/llm-host/package.json packages/llm-host/tsconfig.json packages/llm-host/src/index.ts packages/llm-host/readme.md pnpm-lock.yaml .changeset/llm-host.md README.md docs/ARCHITECTURE.md spec/llm.md docs/specs/llm.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[cloudflare-workers-and-pages]

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status	Deployment URL	Commit	Updated (UTC)	See this deployment's details
Build: Failed ❌ View logs ↗		2995f0e	2026-09-27T21:30:37.210Z	View logs ↗

[HereLiesAz]

OpenCode security review

⚠️ Review unavailable. Neither model answered; see the shared workflow run. This is not a finding about the pull request.

[HereLiesAz]

No validate_* test actually ran on this head. Every one of those checks reports cancelled: the shared CI run was triggered twice for the same commit (once for the push, once for the PR) and the second run cancelled the first. HereLiesAz/workflows#87, now merged, stops that, but only for runs started after it. I can't re-run from here (403), so the next push to this branch, or Re-run failed jobs in the Actions tab, will give a real result.

The other red checks aren't from this PR either: Vercel (a leftover project), Workers Builds (the Cloudflare preview), and github-advanced-security (Copilot). The OpenCode review reports "unavailable" because GEMINI_API_KEY isn't set.

Locally, @azphalt/llm-host passes 20/20 tests, and typecheck and build are clean.

---

Generated by Claude Code

[sourcery-ai]

Sorry @HereLiesAz, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 3 hours by commenting @sourcery-ai review. Upgrade to get a review now.