Skip to content

protocol.md

Latest commit

 

History

History
243 lines (171 loc) · 8.98 KB

protocol.md

File metadata and controls

243 lines (171 loc) · 8.98 KB
Raw

JSON Lines Protocol (Tier 2)

This document defines the communication protocol for Tier 2 process strategies in SpringBot.

Overview

Tier 2 strategies run as managed subprocesses of the SpringBot daemon. Instead of compiling to WASM, you write a standalone program in any language (Python, Node.js, Go, Rust, Java, etc.) that communicates with the daemon over stdin/stdout using newline-delimited JSON (JSON Lines).

Advantages over Tier 1 (WASM):

  • Use any language and any library
  • Full I/O access: load ML models, read files, call external APIs
  • Use existing ecosystems (NumPy, TensorFlow, scikit-learn, etc.)
  • Easier debugging with stderr logging

Tradeoffs:

  • No sandbox -- the process runs with the same OS permissions as the daemon
  • Local only -- process strategies cannot be distributed through the marketplace
  • Slightly higher latency per evaluation (JSON serialization over pipes vs. direct memory access)
  • Resource consumption is not bounded by the engine

Protocol

The daemon and strategy communicate using newline-delimited JSON (JSON Lines) over the process's stdin and stdout:

  • stdin (daemon writes, strategy reads): The daemon sends request messages
  • stdout (strategy writes, daemon reads): The strategy sends response messages
  • stderr (strategy writes): Captured by the daemon and logged with the strategy ID as prefix. Use this for debugging output.

Each message is a single JSON object on one line, terminated by \n. No pretty-printing. No blank lines between messages.

Message Types

Daemon -> Strategy

manifest

Sent once immediately after the process starts. The strategy must respond with its manifest.

{"type": "manifest"}

evaluate

Sent on each tick with the current market data snapshot. The data field contains an EvaluateInput object (see data-types.md) sliced to only the fields your strategy declared in its manifest requires.

{"type": "evaluate", "data": {"candles": [...], "ticker": {...}}}

ping

Health check. The strategy must respond with a pong.

{"type": "ping"}

shutdown

Sent when the bot is stopping. The strategy should clean up resources and exit. After sending this message, the daemon waits up to 5 seconds for the process to exit before killing it.

{"type": "shutdown"}

Strategy -> Daemon

manifest (response)

Response to a manifest request. The data field contains the strategy's manifest.

{"type": "manifest", "data": {"id": "my-strategy", "name": "My Strategy", "version": "1.0.0", "requires": {"candles": {"timeframe": "5m", "lookback": 50}, "ticker": true}}}

See data-types.md for the full manifest schema.

signal (response)

Response to an evaluate request. The data field contains the trading signal.

{"type": "signal", "data": {"direction": "BUY", "confidence": 0.85}}

Valid directions: "BUY", "SELL", "HOLD" (case-sensitive). Confidence must be a float between 0.0 and 1.0 (inclusive).

pong (response)

Response to a ping request.

{"type": "pong"}

Important Rules

  1. One JSON object per line. Do not pretty-print. Do not split a message across multiple lines. Each message must be a single line terminated by \n.

  2. Always flush stdout after writing. Many languages buffer stdout by default when writing to a pipe. If you don't flush, the daemon will never see your response and the evaluate call will time out. Examples:

    • Python: sys.stdout.flush() after each print(), or use print(..., flush=True)
    • Node.js: process.stdout.write() flushes automatically
    • Go: bufio.Writer needs explicit Flush()
    • Java: System.out.flush() or use PrintStream with auto-flush

  3. The process is long-lived. The daemon spawns your process once when the bot starts and keeps it alive across all ticks until the bot stops. Do not exit after handling a single message. Read from stdin in a loop.

  4. Respond to every message. The daemon expects a response for manifest, evaluate, and ping messages. If you don't respond, the call will time out. The only exception is shutdown, which does not expect a response.

  5. Handle shutdown gracefully. When you receive a shutdown message, clean up any resources (close files, connections, etc.) and exit with code 0. If you don't exit within 5 seconds, the daemon will kill the process.

  6. Don't write non-protocol data to stdout. The daemon reads stdout as JSON Lines. If you write debug output, log messages, or anything else to stdout, it will be parsed as (invalid) JSON and cause errors. Use stderr for all non-protocol output.

Timeout Behavior

The daemon uses a 30-second default deadline for each evaluate call. If your strategy does not respond within 30 seconds, the evaluate is treated as a failure. The supervision layer will:

  1. Log the timeout
  2. Return a HOLD signal with confidence 0.0 for that tick
  3. Increment the failure counter

After 3 consecutive failures, the circuit breaker opens and the strategy returns HOLD for a 30-second backoff period. After the backoff, the daemon will attempt to use the strategy again.

The manifest request uses a 10-second timeout.

On-Disk Manifest (manifest.json)

Process strategies must include a manifest.json file in their directory. This file is read by the daemon to determine how to launch the process. It is separate from the protocol manifest response (which the running process sends over stdout).

{
  "id": "my-ml-strategy",
  "name": "My ML Strategy",
  "version": "1.0.0",
  "runtime": "process",
  "command": ["python3", "main.py"],
  "author": "yourname",
  "description": "An ML-based trading strategy using scikit-learn",
  "requires": {
    "candles": { "timeframe": "15m", "lookback": 100 },
    "ticker": true,
    "portfolio": true
  }
}

Required Fields

Field Type Description
id string Unique strategy slug (e.g. "my-ml-strategy"). Must be unique across all installed strategies.
name string Human-readable display name.
version string Semver version string (e.g. "1.0.0").
runtime string Must be "process".
command string[] Argv array for spawning the process. The first element is the executable, the rest are arguments. Executed relative to the strategy directory.
requires object Data requirements. Same schema as the manifest requires field (see data-types.md).

Optional Fields

Field Type Description
author string Strategy author name.
description string Human-readable description of what the strategy does.

Installation

Install a process strategy from a local directory:

springbot strategies install --path ./my-strategy/

The directory must contain a valid manifest.json. The daemon copies the entire directory to its strategy storage location. The command in the manifest is executed relative to the installed directory.

Make sure the command is available on the system. For example, if your strategy uses python3, ensure Python 3 is installed and on the PATH of the user running the daemon.

Example: Minimal Python Strategy

#!/usr/bin/env python3
import sys
import json

def handle_manifest():
    return {
        "type": "manifest",
        "data": {
            "id": "simple-momentum",
            "name": "Simple Momentum",
            "version": "1.0.0",
            "requires": {
                "candles": {"timeframe": "5m", "lookback": 20},
                "ticker": True
            }
        }
    }

def handle_evaluate(data):
    candles = data.get("candles", [])
    if len(candles) < 10:
        return {"type": "signal", "data": {"direction": "HOLD", "confidence": 0.0}}

    # Simple momentum: compare current close to close 10 candles ago
    current = candles[-1]["close"]
    previous = candles[-10]["close"]
    change = (current - previous) / previous

    if change > 0.02:
        return {"type": "signal", "data": {"direction": "BUY", "confidence": min(abs(change) * 10, 1.0)}}
    elif change < -0.02:
        return {"type": "signal", "data": {"direction": "SELL", "confidence": min(abs(change) * 10, 1.0)}}
    else:
        return {"type": "signal", "data": {"direction": "HOLD", "confidence": 0.0}}

def main():
    for line in sys.stdin:
        line = line.strip()
        if not line:
            continue

        msg = json.loads(line)
        msg_type = msg["type"]

        if msg_type == "manifest":
            response = handle_manifest()
        elif msg_type == "evaluate":
            response = handle_evaluate(msg.get("data", {}))
        elif msg_type == "ping":
            response = {"type": "pong"}
        elif msg_type == "shutdown":
            break
        else:
            continue

        print(json.dumps(response), flush=True)

if __name__ == "__main__":
    main()