LLMS.txt

# Harbor Web Agents API - Reference for AI Coding Assistants

> **WHAT THIS IS**: A compact, token-efficient reference for Claude, Cursor, Copilot, or any AI assistant building web apps with the Web Agents API. This doc matches `web-agents-api/src/injected.ts` exactly — if it's not here, it isn't shipped.
>
> **HUMAN DOCS**: For prose explanations, see `docs/WEB_AGENTS_API.md`, `docs/BUILDING_ON_HARBOR.md`, and `docs/AGENTIC_BROWSER_ROADMAP.md`.

---

## TL;DR - Copy This to Start

```javascript
if (!window.agent) throw new Error('Web Agents API not available');

// 1. Feature-detect (synchronous, no permission needed)
const caps = window.agent.capabilities();
if (!caps.features.textGeneration) throw new Error('Text generation disabled');

// 2. Request permissions (REQUIRED before any API use)
const { granted } = await window.agent.requestPermissions({
  scopes: ['model:prompt', 'mcp:tools.list', 'mcp:tools.call'],
  reason: 'Enable AI features'
});
if (!granted) throw new Error('Permissions denied');

// 3. Use text generation
const session = await window.ai.createTextSession({ systemPrompt: 'Be helpful.' });
const response = await session.prompt('Hello!');
session.destroy();

// 4. Or run an autonomous agent (requires toolCalling flag)
if (caps.features.toolCalling) {
  for await (const ev of window.agent.run({ task: 'Search for AI news' })) {
    if (ev.type === 'final') console.log(ev.output);
  }
}
```

---

## Global Objects

| Object | Purpose |
|--------|---------|
| `window.ai` | Text generation (Chrome Prompt API compatible) |
| `window.agent` | Tools, browser control, sessions, multi-agent, MCP, chat |
| `navigator.modelContext` | WebMCP polyfill — page-defined tools |

`window.harbor.ai` and `window.harbor.agent` are unconditional aliases (used when Chrome's native AI is detected and `window.ai` is left untouched).

---

## Detection

```javascript
if (window.agent && window.ai) { /* ready */ }

// Or wait for the ready event
window.addEventListener('agent-ready', (e) => {
  console.log('Version:', e.detail.version);   // '1.4.0'
  console.log('Features:', e.detail.features); // FeatureFlags
});

// Synchronous capability snapshot
const caps = window.agent.capabilities();
// → { apiVersion, features, surfaces, scopes }
```

---

## Permissions (MUST REQUEST FIRST)

All APIs require permission. There are two paths — pick one per session:

### Recommended — `agent.requestCapabilities()` (typed actions)

Mints a session bound to a capability token. Scoped to `plan` /
`execute` / `watch`. Plays nicely with budgets and the activity feed.

```javascript
const session = await window.agent.requestCapabilities({
  name: 'Article assistant',
  reason: 'Read the page and propose follow-ups.',
  mode: 'plan',
  require: [
    { action: 'model.prompt.local' },
    { action: 'browser.read.activeTab' },
    { action: 'tool.call', server: 'time-wasm', toolNames: ['time.now'] },
  ],
  budget: { maxToolCalls: 30, ttlMinutes: 15 },
});

// Narrow into Watch when the user clicks "Run":
await window.agent.upgradeSession(session.id, { mode: 'watch' });
```

### Legacy — `agent.requestPermissions()` (origin-level scopes)

Still works; mapped to typed actions internally.

```javascript
const result = await window.agent.requestPermissions({
  scopes: ['model:prompt', 'mcp:tools.call'],
  reason: 'To help with your task',
  tools: ['time-wasm/time.now']  // Optional allowlist for mcp:tools.call
});

if (result.granted) { /* all scopes granted */ }
else { /* check result.scopes['model:prompt'] === 'granted-always' */ }
```

### Permission Scopes

| Scope | Risk | Enables |
|-------|------|---------|
| `model:prompt` | Low | `ai.createTextSession`, `session.prompt` |
| `model:tools` | Medium | `agent.run` autonomous loop |
| `model:list` | Low | `ai.providers.list/getActive` |
| `mcp:tools.list` | Low | `agent.tools.list` |
| `mcp:tools.call` | Medium | `agent.tools.call` |
| `mcp:servers.register` | Medium | `agent.mcp.register` (BYOC) |
| `browser:activeTab.read` | Medium | `agent.browser.activeTab.readability/getElements` |
| `browser:activeTab.interact` | High | `click`, `fill`, `select`, `scroll` |
| `browser:activeTab.screenshot` | Medium | `agent.browser.activeTab.screenshot` |
| `browser:tabs.read` | Medium | `agent.browser.tabs.list`, `tab.*` reads |
| `browser:tabs.create` | High | `agent.browser.tabs.create/close`, drive spawned tabs |
| `browser:navigate` | High | `agent.browser.navigate` (active tab) |
| `web:fetch` | High | `agent.browser.fetch` (CORS-bypassing) |
| `chat:open` | Low | `agent.chat.open/close` |
| `agents:register` | Medium | `agent.agents.register` |
| `agents:discover` | Medium | `agent.agents.discover/list` |
| `agents:invoke` | Medium | `agent.agents.invoke` |
| `agents:message` | Medium | `agent.agents.send/broadcast` |

### Grant Types

| Grant | Meaning |
|-------|---------|
| `granted-always` | Persistent until revoked |
| `granted-once` | ~10 min or until tab closes |
| `denied` | User said no |
| `not-granted` | Never requested |

```javascript
const { scopes, allowedTools } = await window.agent.permissions.list();
```

---

## Capability Discovery

```javascript
const caps = window.agent.capabilities();
// {
//   apiVersion: '1.4.0',
//   features: { textGeneration, toolCalling, toolAccess,
//               browserInteraction, browserControl, multiAgent },
//   surfaces: {
//     ai, tools, run,
//     browser: { activeTab, tabs, navigate, fetch },
//     mcp, chat, sessions, agents, modelContext
//   },
//   scopes: PermissionScope[]
// }
```

Use `caps.features.*` instead of try/catching `ERR_FEATURE_DISABLED`.

---

## window.ai - Text Generation

```javascript
const session = await window.ai.createTextSession({
  systemPrompt: 'You are helpful.',  // Optional
  temperature: 0.7,                   // Optional
  topK: 40,                           // Optional
});

// Plain prompt
const response = await session.prompt('What is 2+2?');

// Streaming — yields plain string chunks (Chrome Prompt API compatible)
for await (const chunk of session.promptStreaming('Tell me a story')) {
  process.stdout.write(chunk);
}

session.destroy();
```

### Provider listing

```javascript
const providers = await window.ai.providers.list();
const { provider, model } = await window.ai.providers.getActive();
const configured = await window.ai.providers.listConfiguredModels();
```

### Capabilities probe

```javascript
const caps = await window.ai.languageModel.capabilities();
// { available: 'readily' | 'after-download' | 'no', defaultTemperature, ... }
```

> Note: `window.ai.runtime.*` (Firefox/Chrome native AI bridging) is **not** exposed by the Web Agents API extension today. Use `window.ai.providers.list()` and pick a provider explicitly.

---

## window.agent.tools - Direct Tool Access

```javascript
const tools = await window.agent.tools.list();
// [{ serverId: 'time-wasm', name: 'time-wasm/time.now', description, inputSchema }, …]

const result = await window.agent.tools.call({
  tool: 'time-wasm/time.now',
  args: { timezone: 'America/New_York' }
});
```

Page-defined tools (via `navigator.modelContext`) appear with `serverId: 'page'` and a `page/` prefix.

---

## window.agent.run - Autonomous Agent (toolCalling flag)

```javascript
for await (const ev of window.agent.run({
  task: 'What time is it in Tokyo?',
  maxToolCalls: 5,         // default 5
  systemPrompt: 'Be concise.',
})) {
  switch (ev.type) {
    case 'thinking':    console.log('reasoning:', ev.content); break;
    case 'tool_call':   console.log('calling', ev.tool, ev.args); break;
    case 'tool_result': console.log('got', ev.result); break;
    case 'final':       console.log('answer:', ev.output); break;
    case 'error':       throw new Error(ev.error);
  }
}
```

Event union (exhaustive):
```ts
| { type: 'thinking'; content: string }
| { type: 'tool_call'; tool: string; args: Record<string, unknown> }
| { type: 'tool_result'; tool: string; result: unknown }
| { type: 'final'; output: string }
| { type: 'error'; error: string }
```

---

## Browser APIs

### Active tab — `browserInteraction` flag

```javascript
const { title, url, content, length } =
  await window.agent.browser.activeTab.readability();

const { elements } = await window.agent.browser.activeTab.getElements({
  selector: 'a, button, input',
});

await window.agent.browser.activeTab.click('ref-123');
await window.agent.browser.activeTab.fill('ref-456', 'Hello');
await window.agent.browser.activeTab.select('ref-789', 'option-value');
await window.agent.browser.activeTab.scroll('down', 500);

const { dataUrl } = await window.agent.browser.activeTab.screenshot();
```

### Active-tab navigation & fetch — `browserControl` flag

```javascript
// Navigate the page itself
await window.agent.browser.navigate('https://example.com');

// CORS-bypassing fetch (http/https only, blocks RFC1918/localhost)
const r = await window.agent.browser.fetch('https://api.example.com/data', {
  method: 'GET',
  headers: { 'X-Foo': 'bar' },
});
// → { ok, status, statusText, headers, body, url }
```

### Owned tabs — `browserControl` flag

```javascript
const tab = await window.agent.browser.tabs.create({
  url: 'https://flights.example/search',
  active: false,
});
await window.agent.browser.tab.waitForLoad(tab.id);
const { html } = await window.agent.browser.tab.getHtml(tab.id, 'main');
const { title, url, content } = await window.agent.browser.tab.readability(tab.id);
const tabs = await window.agent.browser.tabs.list();
await window.agent.browser.tabs.close(tab.id);
```

You can only drive tabs your origin spawned (same-origin or "owned" model). You cannot drive arbitrary user tabs.

---

## Sessions API (toolAccess flag)

Capability-bounded, named sessions with budgets.

```javascript
const session = await window.agent.sessions.create({
  name: 'Research Assistant',
  reason: 'multi-step research',
  capabilities: {
    llm: { provider: 'ollama', model: 'llama3.2' },
    tools: ['web-search/search', 'memory/save'],
    browser: ['read', 'screenshot'],
  },
  limits: { maxToolCalls: 20, ttlMinutes: 30 },
  options: { systemPrompt: 'Cite sources.', temperature: 0.2 },
});

const response = await session.prompt('Find AI news');
const result = await session.callTool('web-search/search', { query: 'AI' });
const allowed = session.listAllowedTools();
await session.terminate();

const summaries = await window.agent.sessions.list();
```

---

## Multi-Agent API (multiAgent flag)

```javascript
const me = await window.agent.agents.register({
  name: 'Research Agent',
  description: 'Researches and summarizes topics',
  capabilities: ['search', 'summarize'],
  acceptsInvocations: true,
  acceptsMessages: true,
});

window.agent.agents.onInvoke(async (req) => {
  if (req.task === 'search') return { results: [/* … */] };
  throw new Error('Unknown task');
});

window.agent.agents.onMessage((msg) => {
  console.log('from', msg.from, msg.payload);
});

const { agents } = await window.agent.agents.discover({ capabilities: ['analyze'] });
const response = await window.agent.agents.invoke(agents[0].id, {
  task: 'analyze', input: data, timeout: 30000,
});

await window.agent.agents.send(otherId, { hello: 'world' });
await window.agent.agents.broadcast('event-name', { foo: 1 });
```

### Orchestration

```javascript
await window.agent.agents.orchestrate.pipeline({
  steps: [
    { agentId: 'researcher', task: 'research' },
    { agentId: 'writer',     task: 'write' },
  ],
}, initialInput);

await window.agent.agents.orchestrate.parallel({
  tasks: [{ agentId: 'a1', task: 'analyze', input: data }],
  combineStrategy: 'merge',  // 'array' | 'merge' | 'first'
});

await window.agent.agents.orchestrate.route({
  routes: [{ condition: 'type:technical', agentId: 'tech-agent' }],
  defaultAgentId: 'general-agent',
}, input, task);
```

---

## BYOC — `navigator.modelContext` + `agent.mcp` + `agent.chat`

### Page-declared tools (WebMCP polyfill)

```javascript
navigator.modelContext.addTool({
  name: 'add_to_cart',
  description: 'Add a product to the cart',
  inputSchema: { type: 'object', properties: { productId: { type: 'string' } } },
  handler: async ({ productId }) => { /* … */ },
});
```

These show up in `agent.tools.list()` as `page/add_to_cart`.

### Site-hosted MCP server registration

```html
<link rel="mcp-server" href="https://shop.example/mcp"
      title="Shop" data-tools="search,add_to_cart">
```

```javascript
const servers = await window.agent.mcp.discover();

const reg = await window.agent.mcp.register({
  url: 'https://shop.example/mcp',
  name: 'Shop Assistant',
  tools: ['search', 'add_to_cart'],
  transport: 'sse',  // 'sse' | 'stdio' | 'streamable-http'
});

await window.agent.mcp.unregister(reg.serverId);
```

### Open the user's chat with a site-specific config

```javascript
const chat = await window.agent.chat.open({
  systemPrompt: 'Shopping assistant',
  initialMessage: 'How can I help?',
  tools: ['search', 'add_to_cart'],
  style: { theme: 'light', accentColor: '#ff9900', position: 'right' },
});

await window.agent.chat.close(chat.chatId);
```

---

## Error Codes

| Code | Meaning |
|------|---------|
| `ERR_FEATURE_DISABLED` | Feature flag is off (check `agent.capabilities()`) |
| `ERR_NOT_INSTALLED` | Extension not installed |
| `ERR_PERMISSION_DENIED` | User denied or scope not granted |
| `ERR_SCOPE_REQUIRED` | Missing permission scope |
| `ERR_INVALID_REQUEST` | Bad parameters |
| `ERR_TOOL_NOT_ALLOWED` | Tool not in allowlist |
| `ERR_TOOL_NOT_FOUND` | Tool doesn't exist |
| `ERR_TOOL_FAILED` | Tool execution failed |
| `ERR_TOOL_TIMEOUT` | Tool call timed out |
| `ERR_MODEL_FAILED` | LLM error |
| `ERR_SESSION_NOT_FOUND` | Session terminated/destroyed |
| `ERR_TIMEOUT` | Request timed out |
| `ERR_ELEMENT_NOT_FOUND` | DOM ref not found |
| `ERR_AGENT_NOT_FOUND` | Agent not registered |
| `ERR_AGENT_NOT_ACCEPTING` | Agent doesn't accept invocations/messages |
| `ERR_AGENT_FAILED` | Agent invocation threw |
| `ERR_HANDLER_FAILED` | Page handler threw |
| `ERR_INTERNAL` | Internal error |

```javascript
try {
  await window.agent.tools.list();
} catch (err) {
  if (err.code === 'ERR_FEATURE_DISABLED') { /* tell user to enable in sidebar */ }
  if (err.code === 'ERR_PERMISSION_DENIED') { /* fall back */ }
}
```

---

## Feature Flags

Users toggle these in the Web Agents API sidebar. Read them at runtime via `agent.capabilities().features`.

| Flag | Default | Surfaces |
|------|---------|----------|
| `textGeneration` | ON | `window.ai.*` |
| `toolAccess` | ON | `agent.tools.*`, `agent.sessions.*` |
| `toolCalling` | OFF | `agent.run()` |
| `browserInteraction` | OFF | `agent.browser.activeTab.*` |
| `browserControl` | OFF | `agent.browser.navigate/fetch/tabs/tab` |
| `multiAgent` | OFF | `agent.agents.*` |

---

## Debug logging

The injected script is silent by default. Opt in via any of:

```html
<meta name="web-agents-api-debug" content="1">
```

```javascript
localStorage.setItem('web-agents-api:debug', '1');
window.__webAgentsApiDebug = true;  // before injection
```

---

## TypeScript Types (Quick Reference)

```typescript
type PermissionScope =
  | 'model:prompt' | 'model:tools' | 'model:list'
  | 'mcp:tools.list' | 'mcp:tools.call' | 'mcp:servers.register'
  | 'browser:activeTab.read' | 'browser:activeTab.interact' | 'browser:activeTab.screenshot'
  | 'browser:tabs.read' | 'browser:tabs.create' | 'browser:navigate'
  | 'web:fetch' | 'chat:open'
  | 'agents:register' | 'agents:discover' | 'agents:invoke' | 'agents:message';

type PermissionGrant = 'granted-once' | 'granted-always' | 'denied' | 'not-granted';

interface ToolDescriptor {
  serverId: string;
  name: string;
  description?: string;
  inputSchema?: unknown;
}

type RunEvent =
  | { type: 'thinking'; content: string }
  | { type: 'tool_call'; tool: string; args: Record<string, unknown> }
  | { type: 'tool_result'; tool: string; result: unknown }
  | { type: 'final'; output: string }
  | { type: 'error'; error: string };

interface TextSession {
  sessionId: string;
  prompt(input: string): Promise<string>;
  promptStreaming(input: string): AsyncIterable<string>;  // plain strings
  destroy(): void;
}

interface ActiveTabReadability {
  title: string; url: string; content: string; length: number;
}
interface TabInfo { id: number; url: string; title: string; active: boolean; }
interface FetchResponse {
  ok: boolean; status: number; statusText: string;
  headers: Record<string, string>; body: string; url: string;
}
```

---

## Common Patterns

### Initialize app with feature detection + permissions

```javascript
async function init() {
  if (!window.agent) { showError('Install Harbor extension'); return false; }
  const caps = window.agent.capabilities();
  if (!caps.features.textGeneration) { showError('Enable text generation'); return false; }
  const { granted } = await window.agent.requestPermissions({
    scopes: ['model:prompt', 'mcp:tools.list', 'mcp:tools.call'],
    reason: 'Enable AI features',
  });
  return granted;
}
```

### Streaming chat

```javascript
let session;
async function sendMessageStreaming(text, onChunk) {
  session ??= await window.ai.createTextSession({ systemPrompt: 'Be helpful.' });
  for await (const chunk of session.promptStreaming(text)) onChunk(chunk);
}
```

### Page summarizer

```javascript
async function summarizePage() {
  const { content, title } = await window.agent.browser.activeTab.readability();
  const session = await window.ai.createTextSession({
    systemPrompt: 'Summarize in 2-3 sentences.',
  });
  const summary = await session.prompt(content.slice(0, 10000));
  session.destroy();
  return { title, summary };
}
```

### Tool-calling agent

```javascript
async function runAgent(task) {
  const events = [];
  for await (const ev of window.agent.run({ task, maxToolCalls: 5 })) {
    events.push(ev);
    if (ev.type === 'final') return { output: ev.output, events };
    if (ev.type === 'error') throw new Error(ev.error);
  }
}
```

### Graceful degradation

```javascript
async function getAIResponse(prompt) {
  if (window.agent?.capabilities().features.textGeneration) {
    try {
      const { granted } = await window.agent.requestPermissions({ scopes: ['model:prompt'] });
      if (granted) {
        const session = await window.ai.createTextSession();
        const response = await session.prompt(prompt);
        session.destroy();
        return response;
      }
    } catch (e) { console.warn('Web Agents API failed:', e); }
  }
  return await fetch('/api/ai', { method: 'POST', body: JSON.stringify({ prompt }) })
    .then(r => r.json());
}
```

---

## Key Files (for Harbor development)

```
web-agents-api/src/
├── injected.ts            # window.ai/agent/navigator.modelContext (this doc's source of truth)
├── content-script.ts      # Page ↔ background relay
├── background.ts          # Cross-extension bridge to Harbor
└── handlers/              # Per-message handlers (browser, agents, sessions, mcp, chat, tools)

extension/src/             # Harbor core: LLM providers, MCP client, native bridge
bridge-rs/src/             # Native messaging host (Rust)
```

---

## Version

Web Agents API v1.4.0 / Harbor v1