forklift

Forklift trains a tiny Deep-Q-Network to drive 2–3 forklifts around a seed-generated warehouse — pick up packages, deliver them to the dock — and ships a Svelte + TensorFlow.js webapp that runs the trained model in your browser and visualizes the neural network as it decides: every input, hidden activation, and Q-value, live, with per-neuron explanations.

It's deliberately small and honest: an initial, working setup meant to be understood and iterated on, not a state-of-the-art agent. The whole point is that the model is small enough to watch it think.

• 🧠 A 9→12→4 DQN with semantically meaningful inputs — you can read the decision, not just the outcome.
• 👀 Interactive network view — step or play an episode and see activations flow; click any neuron for a plain-language explanation.
• 🔍 Real interpretability — inputs explained by a feature dictionary, outputs by Integrated Gradients, hidden units by gradient × input.
• 🏭 Deterministic, seed-based warehouses — reproducible grids with a difficulty curriculum.
• 🔁 Verified Python ↔ TypeScript parity — the training environment and the browser engine are proven identical by golden fixtures, so the model behaves the same in the browser as it did in training.
• 📦 npm-only host — training runs in Docker (or a local Python 3.12) so you don't need a Python toolchain to use the app.

▶ Live demo: https://mbe24.github.io/forklift/

How it works

Each forklift sees a 9-value observation and the shared network outputs a Q-value per move:

[dx, dy, wall_up, wall_down, wall_left, wall_right, traffic_x, traffic_y, carrying_flag]
        │                         9 sensor inputs
        ▼
   Dense(12, relu)                "forklift_brain_layer"
        ▼
[Q_up, Q_down, Q_left, Q_right]   4 action values → argmax = the move

• Inputs — dx/dy = direction to the current target (a package, or the dock when carrying); wall_* = adjacent shelves/walls; traffic_* = direction to the nearest other forklift; carrying_flag. Offsets use a fixed, tier-independent normalization so the model "sees the same world" on any grid size.
• Reward — +deliver, +pickup, −step, −collision, a −loop penalty that punishes oscillating, and potential-based distance shaping toward the target.
• Training — DQN (experience replay + target network) in Python/Keras, run in Docker. v1 is easy-tier only and produces the top-3 behaviourally-diverse models; medium/hard via warm-start transfer are experimental (the 9-input observation is deliberately myopic).
• Export — trained models are saved directly as browser-ready TensorFlow.js layers-models (model.json + weights), with their validation metrics recorded in a manifest.

Repository layout

forklift/
├── packages/warehouse-generator/  # seed-based deterministic grids (TS) → warehouses.json
├── trainer/                       # Python: env (source of truth), DQN, Docker, TFJS export
│   ├── forklift_env.py            # the parity-critical environment
│   ├── train.py / train_transfer.py
│   └── Dockerfile / requirements.txt
├── webapp/                        # Svelte 5 + Vite + TFJS + Canvas visualization
│   └── src/simEngine.ts           # TS port of the env — parity-tested against the trainer
└── scripts/                       # run-python (local↔Docker), publish-models, lint

Quickstart

Requires Node.js ≥ 20. Then:

npm install
npm run generate     # write the seed-based warehouses.json
npm run dev          # open the webapp (models ship in the repo)

Training

Training needs the pinned scientific stack (tensorflow 2.16, tf-keras, tensorflowjs, numpy — Python 3.9–3.12). scripts/run-python.mjs uses a local trainer/.venv if it can execute, otherwise Docker (python:3.11-slim), so npm is the only hard host dependency.

npm run train           # easy-tier DQN → top-3 diverse models (local venv or Docker)
npm run train:transfer  # warm-start transfer: easy → medium → hard (experimental)
npm run export:tfjs     # publish the trained models into webapp/public/model_output (no Docker)

A local Python 3.12 env can be provisioned with uv: uv venv trainer/.venv --python 3.12 && (cd trainer && uv pip install -r requirements.txt).

npm scripts

Script	What it does
npm run generate	Generate warehouses.json from seeds
npm run dev	Run the webapp dev server
npm run build -w @forklift/webapp	Build the static webapp
npm run train	Train easy tier → top-3 diverse models (local or Docker)
npm run train:transfer	Warm-start transfer easy→medium→hard (Docker)
npm run export:tfjs	Publish models → webapp/public/model_output + manifest
npm run lint:py	pyflakes over the trainer (local, system Python)
npm run test:py	Python env unit tests (pytest)
npm run fixtures	Regenerate the Python↔TS parity fixtures
npm test	All workspace test suites
npm run test:web	Webapp tests incl. simEngine parity
npm run test:gen	Warehouse-generator tests

Development

Fast local loop (all pure-Python/TS, no Docker needed): npm run lint:py, npm run test:py, npm run test:gen, npm run test:web. Only training/export touch TensorFlow, and those run in Docker. CI (.github/workflows/ci.yml) runs the TypeScript and Python checks on every push.

License

Apache 2.0 © Mikael Beyene.