fix: run the HTML phase again when the analysis of its inputs changes

[marcelolynch]

The Lake part of doc-gen4 uses marker files as build products. These files represent data that has been added to the database, because Lake fundamentally tracks files and their contents. Previously, these marker files were empty, which could cause changes upstream of them to not invalidate a build. Now, they contain the hash of everything involved in their production, invalidating them correctly.

Closes: #389

[david-christiansen]

We use squash merges here with the PR description becoming the commit message, so I'm moving Claude's message down to a comment for posterity:

---

Problem

The coreDocs, docInfo, docsHeader and docs steps record their completion in marker files. The comments in the lakefile say that a step which depends on a marker runs again when the trace of the marker changes. The markers are empty files. Lake gives a built file the trace of its content, and an empty file has the same content after every write. So the trace of a marker is the same after every rebuild. The docs step mixes the marker traces of its root modules into its dependency trace. So the docs step stays up to date while the database changes under it.

Example. A library Lib has a page. Add a declaration to a module of Lib and run lake build Lib:docs again.

1. The docInfo step of the module runs, because the oleans of the module changed, and writes the declaration into the database.
2. The step writes its marker again with the same empty content, so the trace of the marker does not change.
3. The docs step sees no change in its inputs and does not run. The page of Lib does not show the new declaration.

Issue #389 reports this behavior: a rebuild after a change to a doc string leaves the page as it was. This PR fixes that part of #389. The other part, a deleted page is not written again, is a separate problem: Lake tracks the marker, not the page.

Change

The lakefile gains writeMarker. It writes the hash of the dependency trace of the step into the marker file, and the four steps call it. The content of a marker then changes exactly when the inputs of its step change. In the example, the marker of the module gets a new hash in step 2. The dependency trace of the docs step changes, and the step writes the page again. With unchanged inputs the content of the marker stays the same, and nothing runs again.

An existing build directory needs no cleanup. A marker gets its hash the first time its step runs again, and the steps that depend on it run at that point.

Test

LibA in test/test-multi-lib-docs.sh imports a second module, LibA.Basic, and the script gains phases 3 to 5.

1. lake build LibA:docs LibB:docs produces the pages of both libraries.
2. lake build LibC:docs produces the page of LibC and keeps the pages of LibA and LibB.
3. The script adds a declaration to the library root LibA and runs lake build LibA:docs again. The page of LibA must show the declaration.
4. lake build LibA:docs --no-build must succeed. A build with no change has nothing to do, so the hash in the marker does not make the HTML phase run every time.
5. The script adds a declaration to LibA.Basic and runs lake build LibA:docs again. The page of LibA.Basic must show the declaration, and the pages of the other two libraries must remain. This change reaches the docs step through two markers.

Phases 3 and 5 fail on main. Phase 4 passes on main and guards against a marker whose content changes while the inputs stay the same.

🤖 Generated with Claude Code