Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ Thumbs.db
.vscode
.env
.superpowers/
__pycache__/
17 changes: 16 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

## [Unreleased]

## [1.2.0] - 2026-09-09

### Added

- **A baseline, which is what makes `vacuum:lint` usable on a schema that predates it.** The first run against a five-year-old application prints several hundred findings, and the only available response was to stop running it. `--generate-baseline` writes `vacuum-baseline.json`, you commit it, and from then on the outstanding findings are excused and anything new fails the build. A finding is matched on its rule and its subject and on nothing else — not the prose, not the severity — so rewording a rule never invalidates a file somebody committed months ago. That is defensible only because schema-rule subjects are data-independent: `public.orders.customer_id` means the same thing on every run, which is not true of `slow-statement` and is why `vacuum:check` has no baseline. Entries that stop matching are reported as `Info` rather than silently carried, because a baseline nobody prunes becomes a place the next defect hides, and the score is computed over what is left with the suppressed count printed beside it.

- **`--format=github`, so findings arrive on the pull request rather than in a log.** Each becomes a workflow-command annotation on the diff, and a markdown table is appended to `$GITHUB_STEP_SUMMARY` when the runner offers one.

- **Findings are traced back to the migration that introduced them.** `database/migrations` is read with PHP's own tokenizer — the technique the Filament installer already uses, and with the same refusal to guess: a variable table name or a file that does not parse yields no anchor, and the finding is reported without one rather than pointed at a line it did not come from. An application that has run `schema:dump --prune` has little left to trace to — that flag is the one that deletes the migrations after squashing them, while plain `schema:dump` keeps them and is unaffected — which the README says plainly rather than leaving to be discovered.

### Changed

- **`int4-primary-key` is now `narrow-primary-key`.** The slug named a type rather than the defect, and the rule has always fired on `smallint` as well — it prints 32,767 or 2,147,483,647 as appropriate. The rename is deliberate and deliberately early: the slug is about to become a key in the baseline file users commit, and renaming it once anybody has a baseline would invalidate all of them. It appears as the `rule` value in the `vacuum:lint --format=json` document, so a pipeline filtering on the old string needs updating.

## [1.1.0] - 2026-09-08

### Added
Expand Down Expand Up @@ -138,7 +152,8 @@ First release.
- **A Filament v4 panel** (optional peer — nothing changes for a Blade-only install): a **Vacuum** navigation group with an **Overview** dashboard (health score and grade, database vitals, charts, the findings with copyable remediation, and live running vacuums) and read-only resources for **Tables**, **Indexes**, **Sessions** and **Statements**. Every surface shares the one `Vacuum::auth()` gate and opts out of tenant scoping, so it is at home in a multi-tenant panel.
- **Extensibility.** Application rules can be tagged onto the advisor per subject (`TABLE_RULES`, `INDEX_RULES`, and the rest), and both the config and the dashboard views are publishable.

[Unreleased]: https://github.com/heyosseus/vacuum/compare/v1.1.0...HEAD
[Unreleased]: https://github.com/heyosseus/vacuum/compare/v1.2.0...HEAD
[1.2.0]: https://github.com/heyosseus/vacuum/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/heyosseus/vacuum/compare/v1.0.1...v1.1.0
[1.0.0]: https://github.com/heyosseus/vacuum/compare/v0.3.0...v1.0.0
[0.3.0]: https://github.com/heyosseus/vacuum/compare/v0.1.0...v0.3.0
Expand Down
82 changes: 76 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,21 @@

![Vacuum — a PostgreSQL monitoring and tuning dashboard for Laravel](art/hero.png)

**A PostgreSQL monitoring and tuning dashboard for Laravel.**
**A PostgreSQL monitoring dashboard for Laravel — and a schema linter for the pipeline that ships it.**

Vacuum reads what PostgreSQL already knows about itself — `pg_stat_user_tables`, `pg_stat_user_indexes`, `pg_stat_activity`, `pg_stat_database`, `pg_stat_statements`, `pg_class` — and turns it into a page that says what is wrong, what it is costing you, and the statement that would put it right.

It shows you that statement. It never runs it.

> **Status: 1.1.0.** The public API is frozen — see [What semver covers](#what-semver-covers). A breaking change to the rule contracts, `Finding`, the value objects, the configuration keys or the `--format=json` documents now requires a major version.
**It does not need a production database to be useful.** `vacuum:lint` reads the catalog rather than the statistics, so it has something to say against the empty Postgres container your test job already starts: a foreign key with no index behind it, a primary key that stops accepting rows at two billion, a table nothing can address a single row of. One line in the workflow you already have —

```yaml
- run: php artisan vacuum:lint --format=github
```

— and the finding arrives as an annotation on the pull request that introduced it, on the line that introduced it. See [Linting the schema](#linting-the-schema).

> **Status: 1.2.0.** The public API is frozen — see [What semver covers](#what-semver-covers). A breaking change to the rule contracts, `Finding`, the value objects, the configuration keys or the `--format=json` documents now requires a major version.

## Quick start

Expand All @@ -31,7 +39,9 @@ Then open `/vacuum`. That is all of it: the installer publishes the config and a

![How Vacuum works: six catalogs PostgreSQL maintains, thirteen rules, and a finding carrying the statement that fixes it](art/how-it-works.png)

Already running a Filament panel? `php artisan vacuum:install --filament` puts the same data inside it — see [Inside Filament](#inside-filament). Want it in CI instead of in a browser? `php artisan vacuum:check` — see [In your pipeline](#in-your-pipeline).
Already running a Filament panel? `php artisan vacuum:install --filament` puts the same data inside it — see [Inside Filament](#inside-filament).

**In a pipeline there are two commands, and they answer different questions.** `vacuum:check` runs the full advisor against a database that has been *running* — see [In your pipeline](#in-your-pipeline). `vacuum:lint` runs against one that has only been *migrated*, which is what a test job actually has — see [Linting the schema](#linting-the-schema). The first belongs on a schedule against staging; the second belongs in `require-dev`, on every push.

## Contents

Expand All @@ -41,7 +51,7 @@ Already running a Filament panel? `php artisan vacuum:install --filament` puts t
- [The standalone dashboard](#the-standalone-dashboard) · [Inside Filament](#inside-filament)
- [Who may look](#who-may-look) · [Which database](#which-database)
- [In your pipeline](#in-your-pipeline) — `vacuum:check`, and failing a build
- [Linting the schema](#linting-the-schema) — `vacuum:lint`, and what is wrong before a row exists
- [Linting the schema](#linting-the-schema) — `vacuum:lint`, a baseline, and annotations on the pull request
- [History over time](#history-over-time) — direction, forecasts, and what changed
- [The SQL console](#the-sql-console) — and what actually makes it safe
- [Tuning the thresholds](#tuning-the-thresholds) · [Writing your own rule](#writing-your-own-rule) · [Restyling the dashboard](#restyling-the-dashboard)
Expand Down Expand Up @@ -231,7 +241,7 @@ php artisan vacuum:check --format=json # score, grade, deductions, finding

Two things worth knowing. It **never writes** — the remediation is printed for you to read and decide on, exactly as it is on the page. And if Vacuum is disabled it **fails rather than passing**: a check that goes green because it never looked is worse than no check at all.

### Linting the schema
## Linting the schema

`vacuum:check` reads what the database has been *doing*, and a pipeline's database
has not done anything. A Postgres container ninety seconds old with the migrations
Expand All @@ -245,11 +255,13 @@ ever used is exactly the kind of green number this package exists to argue again
php artisan vacuum:lint
```

![vacuum:lint in a pipeline: the workflow step on the left, and the finding as an annotation on the pull request diff on the right](art/lint-in-ci.png)

| Rule | Finds |
| --- | --- |
| `unindexed-foreign-key` | Foreign keys PostgreSQL created no index for, which `->constrained()` never does |
| `foreign-key-type-mismatch` | A key referencing a different type, so the index exists and cannot be used |
| `int4-primary-key` | A primary key that stops accepting rows at 2,147,483,647 |
| `narrow-primary-key` | A primary key too narrow to keep counting -- `integer` or `smallint` -- that stops accepting rows the moment it runs out of values |
| `missing-primary-key` | Tables nothing can address a single row of |
| `unindexed-morphs` | A polymorphic pair with no composite index leading on the type |
| `json-not-jsonb` | `json` where `jsonb` was almost certainly meant |
Expand All @@ -272,6 +284,61 @@ health score. Adding rules to that score would silently re-grade every existing
installation on a `composer update`, and a grade that moves for a reason nobody
asked for is worse than one rule fewer.

### Adopting it on a schema that predates it

Run `vacuum:lint` on a five-year-old application and it will find a great many
things. That is accurate and completely useless: nobody is going to fix four
hundred findings this afternoon, and a build that is red for reasons nobody
intends to act on is a build people learn to ignore.

So write down what is already there, and let the linter tell you only what is new:

```bash
php artisan vacuum:lint --generate-baseline
```

That writes `vacuum-baseline.json`. **Commit it.** From then on the outstanding
findings are excused and anything new fails the build, which is the only question
worth asking of a legacy schema.

A baseline matches on the rule and the subject and on nothing else, so rewording a
rule — or making it more serious in a later release — never invalidates the file
you committed. When an entry stops matching anything, because somebody fixed it,
`vacuum:lint` says so as an `Info` finding rather than quietly carrying it: a
baseline nobody prunes becomes a place the next defect hides.

```bash
php artisan vacuum:lint --no-baseline # report everything, baseline or not
php artisan vacuum:lint --baseline=path # somewhere other than the default
```

The score is computed over what is left after suppression, and the count of what was
suppressed is printed with the text output, carried as `suppressed` in the JSON document,
and emitted as a `::notice` for `--format=github`. A number that quietly ignored four
hundred findings would be the kind of green this package exists to argue against —
and the pull request is the one place that number matters most.

### On the pull request

`--format=github` emits GitHub Actions workflow commands, so each finding lands as
an annotation on the diff rather than in a log nobody opens.

```yaml
- run: php artisan vacuum:lint --format=github
```

Findings are traced back to the migration that introduced them by parsing
`database/migrations` with PHP's own tokenizer — the same technique the Filament
installer uses, and with the same refusal to guess. A migration whose table name is
a variable, or that does not parse, yields no anchor; the finding is still
reported, without a file and a line.

**If you have run `php artisan schema:dump --prune`, expect few anchors.** That flag deletes
`database/migrations` after squashing the schema into `database/schema/*.sql`, so the files
that declared your columns are gone and there is nothing left to trace to. Plain
`schema:dump` keeps the migrations and is unaffected. The findings are the same either way;
only the annotations lose their line numbers.

## History over time

Vacuum is point-in-time by default: every page and every `vacuum:check` reads the database as it is this instant. Switch history on and it records a snapshot on a schedule, so it can tell you which way a number is *moving* — bloat that is growing, a freeze age climbing since the last time anything froze it, a cache-hit ratio measured over the last hour rather than over the life of the server.
Expand Down Expand Up @@ -436,13 +503,16 @@ From 1.0 — and from 1.1 where a line says so — these are public API and a br
- **The `vacuum:check --format=json` and `vacuum:lint --format=json` documents**, which are what a pipeline parses.
- **Route names** (`vacuum.dashboard` and the rest) and the `Vacuum::auth()` gate.
- **The `SCHEMA_RULES` tag** (1.1), alongside the others a custom rule is registered under.
- **The baseline file format** (1.2). It is committed to your repository, which makes it an interface whether or not it is called one. Keys may be added; the `findings` map will not change meaning.
- **`vacuum:lint --format=github`** (1.2) as an accepted value, and its severity mapping. The exact wording of an annotation is not covered.

Explicitly **not** covered, and free to change in a minor release:

- The SQL files under `resources/sql`. They are readable on purpose and they are not an interface — a catalog query is rewritten whenever PostgreSQL gives a better way to ask.
- The Blade views. Publishing them is supported; the markup inside them is not frozen.
- Everything under `Internals` and `Learn`. Both are teaching surfaces, and pinning their shape would freeze the explanation as well as the code.
- Anything marked `@internal`.
- `MigrationMap` and `SourceLocation`, which are console implementation detail rather than something a rule or a pipeline consumes.

Vacuum supports the PostgreSQL major versions the PostgreSQL project still supports, and CI runs the suite against each of them. A major going end-of-life is a minor release here, not a major one.

Expand Down
Binary file added art/lint-in-ci.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
143 changes: 143 additions & 0 deletions art/make_readme_art.py
Original file line number Diff line number Diff line change
Expand Up @@ -580,9 +580,152 @@ def cli_check() -> None:
a.save("cli-check.png")


# ---------------------------------------------------------------------------
# 6. vacuum:lint in a pipeline
# ---------------------------------------------------------------------------

WORKFLOW = [
("services:", DIM),
(" postgres: { image: postgres:17 }", GRAY),
("", None),
("steps:", DIM),
(" - run: php artisan migrate --force", GRAY),
(" - run: php artisan vacuum:lint --format=github", WHITE),
]

# The diff as a reviewer sees it: the line that introduced the finding is the
# line the finding lands on.
DIFF = [
(" ", "12", "Schema::create('orders', function (Blueprint $table) {", GRAY, False),
("+", "13", " $table->id();", GRAY, True),
("+", "14", " $table->foreignId('customer_id');", WHITE, True),
(" ", "15", "});", GRAY, False),
]


def lint_pr() -> None:
W, H = 1600, 620
a = Art(W, H)

a.text(
W / 2,
48,
"The database is ninety seconds old and has no rows in it. "
"The finding still lands on the line that caused it.",
size=17,
fill=GRAY,
anchor="mm",
)

top, ph = 108, 400
ax, aw = 70, 520
bx, bw = 640, 890

a.panel(ax, top, aw, ph)
a.panel(bx, top, bw, ph)
a.arrow(600, top + ph / 2, 632, colour=DIM)

# --- A: the workflow
y = top + 26
a.text(ax + 24, y, "IN YOUR WORKFLOW", size=12, bold=True, fill=GRAY)
y += 36
a.rect(ax + 24, y, aw - 48, 172, r=8, fill=(14, 15, 18), outline=LINE, width=1)
ly = y + 20
for line, colour in WORKFLOW:
if line:
a.text(ax + 40, ly, line, size=13, fill=colour)
ly += 25
y += 196

a.chip(ax + 24, y, "require-dev", TEAL, size=12)
y += 46
for ln in (
"No production database, no credentials, no",
"extension and no superuser. Every rule here",
"is answerable the moment migrate finishes.",
):
a.text(ax + 24, y, ln, size=13.5, fill=DIM)
y += 22

# --- B: the pull request
y = top + 26
a.text(bx + 24, y, "ON THE PULL REQUEST", size=12, bold=True, fill=GRAY)
y += 30
a.text(
bx + 24,
y,
"database/migrations/2024_01_11_000000_create_orders_table.php",
size=13,
fill=DIM,
)
y += 30

for mark, number, code, colour, added in DIFF:
if added:
a.rect(bx + 24, y - 4, bw - 48, 26, r=4, fill=(*TEAL, 16))
a.text(bx + 34, y, number, size=13, fill=(70, 74, 82))
a.text(bx + 72, y, mark, size=14, bold=True, fill=TEAL if added else DIM)
a.text(bx + 92, y, code, size=14, fill=colour)
y += 26

# the annotation, hung under the line that produced it
y += 16
ah = 152
a.rect(bx + 92, y, bw - 140, ah, r=8, fill=(14, 15, 18), outline=(*AMBER, 90), width=1)
a.rect(bx + 92, y, 4, ah, r=2, fill=AMBER)

iy = y + 20
cw = a.chip(bx + 116, iy, "WARNING", AMBER, size=11)
a.text(bx + 116 + cw + 14, iy + 12, "unindexed-foreign-key", size=12.5, fill=DIM, anchor="lm")
iy += 40
a.text(
bx + 116,
iy,
"orders.customer_id has a foreign key and no index behind it.",
size=14,
fill=WHITE,
)
iy += 26
a.text(
bx + 116,
iy,
"PostgreSQL indexes a primary key and creates nothing for this.",
size=13,
fill=GRAY,
)
iy += 30
a.rect(bx + 116, iy, bw - 188, 32, r=6, fill=(21, 23, 28), outline=LINE, width=1)
a.text(
bx + 128,
iy + 16,
'CREATE INDEX CONCURRENTLY ON "public"."orders" ("customer_id");',
size=12,
fill=TEAL,
anchor="lm",
)

# --- the verdict
by, bh = 526, 62
a.rect(70, by, 1460, bh, r=10, fill=(*CORAL, 20), outline=(*CORAL, 70), width=1)
a.rect(70, by, 4, bh, r=2, fill=CORAL)
a.text(
100,
by + bh / 2,
"Exit 1. A warning from vacuum:check is a database drifting; a warning from "
"vacuum:lint is a schema that was wrong the moment somebody typed it.",
size=17,
bold=True,
fill=CORAL,
anchor="lm",
)

a.save("lint-in-ci.png")


if __name__ == "__main__":
hero()
how_it_works()
scoring()
safety()
cli_check()
lint_pr()
25 changes: 25 additions & 0 deletions config/vacuum.php
Original file line number Diff line number Diff line change
Expand Up @@ -309,4 +309,29 @@
'enabled' => env('VACUUM_LEARN_ENABLED', true),
],

/*
|--------------------------------------------------------------------------
| Lint
|--------------------------------------------------------------------------
|
| vacuum:lint reads the shape of the schema rather than its statistics, so it
| has something to say in a pipeline where every statistics-based rule finds
| nothing. These two keys are what it needs from the filesystem.
|
| 'baseline' is resolved against base_path() and is used automatically when
| the file exists. It is what makes the linter adoptable on a schema that
| predates it: without one, the first run on a legacy application prints
| several hundred findings and the only available response is to stop running
| it.
|
| 'migrations_path' is where findings are traced back to the line that
| introduced them. Null means database_path('migrations').
|
*/

'lint' => [
'baseline' => env('VACUUM_LINT_BASELINE', 'vacuum-baseline.json'),
'migrations_path' => env('VACUUM_LINT_MIGRATIONS_PATH'),
],

];
Loading