demo-micrograd

A tiny scalar reverse-mode autodiff engine written in Jennifer, in the spirit of Andrej Karpathy's micrograd - built as a standalone example of a real Jennifer project.

It is written to be read in order. Each edition lives in its own directory and is a complete, runnable project; the interesting part is the diff from the one before it.

Editions

Edition	Tag	What it adds
v1/	1.0.0	the engine, deliberately minimal: an arena / tape, Leaf Add Sub Mul Tanh, backward, sgd
v2/	2.0.0	more ops: pow, relu (math.max(0, x)), exp / ln (both in math), and the constant-operand field pow needs
v3/	3.0.0	a neuron and an MLP: a neuron is tanh(sum(w_i * x_i) + b); a layer is a list of neurons; a 2-layer MLP is the classic micrograd finale. The parameters are the leading leaf nodes; resetTo(numParams) rebuilds the graph each step
v4/	4.0.0	batching and a real dataset: two-moons classification with mini-batch SGD, a margin loss and L2
v5/	5.0.0	making it fast: profile first, then stop zeroing the whole tape in backward, fold value op constant into one node, and apply L2 as weight decay instead of building it as graph. 17% faster, byte-identical output - and two predicted wins that measured at zero

Run one

cd v1
jennifer run main.j             # the two demos
jennifer test autograd_test.j   # the gradient-check tests

$ jennifer run main.j
Demo 1 - loss = tanh(x*w + b)   [x=2, w=-3, b=1]
  loss     = -0.999909
  dloss/dx = -0.000545   (check -0.000545)
  ...
Demo 2 - fit y = 2x + 1 with SGD
  epoch 0:   loss=164.000000  w=0.700000  b=0.240000
  ...
  final:      w=2.008823  b=0.974060   (target w=2, b=1)

Each edition has its own README explaining what it does and why.

Read the diffs

The step from one edition to the next is the lesson, so read it as a diff:

git diff --no-index v1 v2
git diff --no-index v1/autograd.j v2/autograd.j   # just the engine

Every edition lands as two commits: one that copies the previous edition verbatim, then one that changes it. That second commit is the diff above, so it can be linked and reviewed on its own.

Why directories and not branches

Editions are directories on main rather than long-lived branches, so that

• every edition is tested by CI on every push, against a pinned Jennifer release. A second, weekly job runs them all against the language's main as well: Jennifer is pre-1.0 and takes breaking changes at milestone boundaries, so an old edition can break through no fault of its own. That job is allowed to fail - a red one means an edition needs updating, not that a pull request is broken. Better a red weekly check than a learner hitting a parse error;
• a change that belongs in all editions (a doc convention, a license header) is one commit, not one cherry-pick per branch;
• an edition can still be fixed independently forever: edit v1/, tag 1.0.1, and the others are untouched.

Editions are tagged N.0.0 when they land. The major number is the edition number - a chapter marker, not an API-compatibility claim.

Repository layout

Path	What
v1/ ...	one self-contained edition per directory
JENNIFER.md	the Jennifer language reference, for coding assistants
.github/	CI: installs the pinned Jennifer release, runs every edition in a matrix

License

MIT.