Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,26 +55,26 @@ jobs:
- name: Test
run: go test -race ./...

# The bench path: the same binary provisions the service container the
# way it provisions production. Run twice to prove idempotency. "all"
# ends with verify, which reports "writer has not run" as a state, not
# a failure - the expected order before any SCADA has started.
# The site path, the only thing that covers the ALTER SYSTEM tuning: the
# same binary provisions the service container the way it provisions an
# installation machine. Run twice to prove idempotency - the second run
# finds every object in place and still exits on a passing access check.
- name: Provision the service container
env:
SEMIBASE_SUPER_PASSWORD: ci-super
SEMIBASE_WRITER_PASSWORD: ci-writer
SEMIBASE_READER_PASSWORD: ci-reader
run: |
go build -o semibase ./cmd/semibase
./semibase all --database semiplot_dev --expected-major 17
./semibase all --database semiplot_dev --expected-major 17
./semibase site --database semiplot_dev --expected-major 17
./semibase site --database semiplot_dev --expected-major 17

# The socket path, which nothing else covers: the same binary, run the way
# the official image runs an init script. During init the server has
# listen_addresses empty and only /var/run/postgresql answers, superuser
# auth is local trust, and the script's own "set -e" makes the entrypoint
# abort on failure. So the container reaching a state where semiplot_reader
# selects over TCP proves create completed over the socket.
# selects over TCP proves bench completed over the socket.
- name: Provision over a unix socket, as a postgres image init script
env:
SEMIBASE_WRITER_PASSWORD: ci-writer
Expand All @@ -96,7 +96,7 @@ jobs:
cat > initdb-semibase.sh <<'SCRIPT'
#!/bin/sh
set -e
/semibase create --host /var/run/postgresql --database semiplot_dev --expected-major 17
/semibase bench --host /var/run/postgresql --database semiplot_dev --expected-major 17
SCRIPT
# the entrypoint sources a non-executable init script instead of running it,
# and a binary without the bit set cannot run at all
Expand Down
46 changes: 29 additions & 17 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ provisions and configures the instance that hosts the Simple-Scada 2 archive: th
CLI binary, `semibase.exe`, a container image carrying its Linux build, plus the instance's
architecture docs.
Deployment target: Windows. Language: Go (module `github.com/Semiteq/SemiBase`), driver
`pgx/v5`, `sql/semiplot_tags.sql` embedded via `go:embed`. Entry point: `cmd/semibase`.
The module is pure `pgx` and compiles for any GOOS; Linux builds serve the containerised
test bench of consumers (SemiPlot runs `create` against an ephemeral `postgres:17` container).
All commands run from the repository root.
`pgx/v5`, `sql/semiplot_tags.sql` and `sql/trends.sql` embedded via `go:embed`. Entry point:
`cmd/semibase`. The module is pure `pgx` and compiles for any GOOS; Linux builds serve the
containerised test bench of consumers (SemiPlot runs `bench` against an ephemeral `postgres:17`
container). All commands run from the repository root.

## Build

Expand Down Expand Up @@ -80,13 +80,15 @@ golangci-lint run
golangci-lint is pinned to 2.12.2 (`winget install GolangCI.golangci-lint --version 2.12.2`);
config in `.golangci.yml`. CI (`.github/workflows/ci.yml`) runs `go build`, `go test -race`,
and the same lint on `windows-latest` and `ubuntu-latest` for every push and pull request;
the Linux job also provisions a `postgres:17-alpine` service container by running `all`
twice — the bench path and the idempotency check in one step — and builds the container image,
so a broken `Dockerfile` fails on the pull request rather than at tag time. CI pushes nothing.
the Linux job also provisions a `postgres:17-alpine` service container by running `site`
twice — the tuning path and the idempotency check in one step — provisions a second container
with `bench` as a `postgres` image init script over the unix socket, and builds the container
image, so a broken `Dockerfile` fails on the pull request rather than at tag time. CI pushes
nothing.

Unit tests live beside the source (`cmd/semibase/*_test.go`, `internal/provision/*_test.go`),
table-driven. There are no database-touching tests; the `verify` command is the integration
check, run against a live server.
table-driven. There are no database-touching tests in the repository; the integration check is
the reader-access check both commands run at the tail, against a live server.

## Format

Expand All @@ -97,24 +99,31 @@ gofmt -w . # run before presenting changes; gofmt is authoritative
## Run

```powershell
.\semibase.exe --help # commands: config | create | verify | all | version
.\semibase.exe create --port 15432 --database semiplot_dev --expected-major 17
.\semibase.exe verify --reader-password <pw> # post-writer proof of the reader access chain
.\semibase.exe --help # commands: site | bench | version
.\semibase.exe site # an installation machine: tuning, then everything else
.\semibase.exe bench --port 15432 --database semiplot_dev --expected-major 17
.\semibase.exe version # print the build revision
```

`site` and `bench` differ in one thing: `site` applies the `ALTER SYSTEM` memory constants,
`bench` does not. Both create the database, the roles, the grants, `semiplot_tags` and
`public.trends`, and both end by actually reading `public.trends` as `semiplot_reader` and
checking that the same role holds no `INSERT`. A failed check is a non-zero exit.

Passwords come from flags, env, or a `.env` file in the working directory (flag > env > `.env`;
template `.env.example`): `SEMIBASE_SUPER_PASSWORD`, `SEMIBASE_WRITER_PASSWORD`,
`SEMIBASE_READER_PASSWORD`.
`verify` failing with "writer has not run" before the SCADA's first start is the expected
order, not a bug.
`SEMIBASE_READER_PASSWORD`. The writer and reader passwords are needed only on a first run,
which creates those roles; `public.trends` is created under `SET ROLE scada_writer` on the
superuser connection, so the table's owner is the role the SCADA writes with and no
`pg_hba.conf` line has to admit a `scada_writer` login.

## Layout

```
cmd/semibase/ CLI entry point: command dispatch, flags, usage text
internal/provision/ the phases: config (ALTER SYSTEM), create (db/roles/grants), verify
sql/ DDL for objects we own (semiplot_tags), embedded into the binary
internal/provision/ the phases behind the two commands: config (ALTER SYSTEM), create
(db/roles/grants/semiplot_tags/trends), check (reader access)
sql/ embedded DDL: semiplot_tags (ours) and trends (the vendor's shape)
docs/ human docs in Russian (enter at docs/readme.md)
docs/architecture/ agent-facing design docs in English
docs/plans/ dated implementation plans
Expand All @@ -129,3 +138,6 @@ docs/plans/ dated implementation plans
in `docs/plans/completed/`.
- The archive schema itself belongs to the SCADA and is documented in the SemiPlot repository
(`docs/architecture/scada-archive.md` there); this repository documents the instance around it.
`sql/trends.sql` is a transcription of the vendor's shape, created once and never altered —
whether the SCADA tolerates finding it already there is an unverified assumption, stated with
its experiment in `docs/architecture/provisioning.md`.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ SemiBase — разворачиваемый PostgreSQL-сервис для ус
# установка pgsql
winget install --id PostgreSQL.PostgreSQL.17 --exact

# конфигурация сервера, архивная база, роли, права
.\semibase.exe all
# конфигурация сервера, архивная база, роли, права, таблица архива
.\semibase.exe site
```
32 changes: 19 additions & 13 deletions cmd/semibase/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,22 @@ import (
"github.com/Semiteq/SemiBase/internal/provision"
)

const usage = `semibase - provisions the SemiBase PostgreSQL instance.
const usage = `semibase - brings a PostgreSQL instance to its provisioned state and exits.

Usage:
semibase <command> [flags]

Commands:
config apply the server configuration deltas (ALTER SYSTEM + reload);
shared_buffers takes effect at the next service restart
create create the archive database, the roles, the access chain, and semiplot_tags
verify post-writer checks: archive tables exist, the reader reads and cannot write
all config + create + verify
site an installation machine: apply the memory tuning (ALTER SYSTEM + reload),
then create the archive database, the roles, the access chain,
semiplot_tags and public.trends
bench a throwaway container: the same, without the tuning
version print the build revision

Every step checks before it creates; re-running any command is safe. Passwords of
Both commands end by reading public.trends as semiplot_reader and checking that the
same role cannot write it; a failed check is a non-zero exit.

Every step checks before it creates; re-running either command is safe. Passwords of
existing roles change only when the corresponding flag or variable is set.

Passwords come from flags, environment variables, or a .env file in the working
Expand All @@ -39,6 +41,10 @@ directory (flag wins over environment, environment wins over .env):
--writer-password SEMIBASE_WRITER_PASSWORD scada_writer
--reader-password SEMIBASE_READER_PASSWORD semiplot_reader

A first run creates both roles and fails without their passwords. Once the roles exist,
the superuser password alone carries a run; the reader password then only decides
whether the run also tests the semiplot_reader login, which it does over TCP.

Run 'semibase <command> --help' for the command's flags.
`

Expand All @@ -55,10 +61,8 @@ var (

// only tests write to this map, to insert stubs
var commands = map[string]func(context.Context, provision.Options) error{
"config": func(ctx context.Context, options provision.Options) error { return options.Config(ctx) },
"create": func(ctx context.Context, options provision.Options) error { return options.Create(ctx) },
"verify": func(ctx context.Context, options provision.Options) error { return options.Verify(ctx) },
"all": func(ctx context.Context, options provision.Options) error { return options.All(ctx) },
"site": func(ctx context.Context, options provision.Options) error { return options.Site(ctx) },
"bench": func(ctx context.Context, options provision.Options) error { return options.Bench(ctx) },
}

func main() {
Expand Down Expand Up @@ -141,8 +145,10 @@ func revisionFromSettings(settings []debug.BuildSetting) string {
return vcsRevision
}

// password flags default to empty so the usage output can only ever show the
// variable names, never their values
// One flag set serves both commands: site and bench differ only in the tuning phase,
// which reads no flag of its own, so every flag below is used by either command.
// Password flags default to empty so the usage output can only ever show the variable
// names, never their values.
func newFlagSet(command string, options *provision.Options) *flag.FlagSet {
flags := flag.NewFlagSet(command, flag.ContinueOnError)
flags.SetOutput(errorOutput)
Expand Down
30 changes: 23 additions & 7 deletions cmd/semibase/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ func TestFlagUsageHidesPasswordValues(t *testing.T) {
}

options := provision.Options{}
flags := newFlagSet("create", &options)
flags := newFlagSet("bench", &options)
buffer := &bytes.Buffer{}
flags.SetOutput(buffer)
flags.PrintDefaults()
Expand Down Expand Up @@ -134,7 +134,7 @@ func TestParseOptions(t *testing.T) {
t.Setenv(name, value)
}

options, err := parseOptions("create", tt.arguments)
options, err := parseOptions("bench", tt.arguments)

switch {
case tt.wantErr != nil:
Expand Down Expand Up @@ -221,6 +221,22 @@ func TestRunVersionCommandExitsZero(t *testing.T) {
}
}

// the surface is two commands and nothing else: config, create, verify and all were
// removed with no alias and no tombstone
func TestCommandSurface(t *testing.T) {
want := map[string]bool{"site": true, "bench": true}
for name := range commands {
if !want[name] {
t.Errorf("unexpected command %q in the dispatch map", name)
}
}
for name := range want {
if _, known := commands[name]; !known {
t.Errorf("command %q is missing from the dispatch map", name)
}
}
}

func TestRunUnmappedCommandIsAnError(t *testing.T) {
buffer := captureErrorOutput(t)

Expand All @@ -247,8 +263,8 @@ func TestRunHelpFlagExitsZero(t *testing.T) {
buffer := captureErrorOutput(t)
clearPasswordEnvironment(t)

if got := run(context.Background(), []string{"create", "--help"}); got != 0 {
t.Fatalf("run(create --help) = %d, want exit code 0", got)
if got := run(context.Background(), []string{"bench", "--help"}); got != 0 {
t.Fatalf("run(bench --help) = %d, want exit code 0", got)
}
if !strings.Contains(buffer.String(), "-super-password") {
t.Errorf("help output does not list the command flags: %q", buffer.String())
Expand Down Expand Up @@ -336,13 +352,13 @@ func TestCancelledContextReturnsPromptly(t *testing.T) {
cancel()

started := time.Now()
err := commands["create"](ctx, options)
err := commands["bench"](ctx, options)
elapsed := time.Since(started)

if !errors.Is(err, context.Canceled) {
t.Fatalf("create with cancelled context returned %v, want context.Canceled in the chain", err)
t.Fatalf("bench with cancelled context returned %v, want context.Canceled in the chain", err)
}
if elapsed > 2*time.Second {
t.Errorf("create with cancelled context took %v, want a prompt return", elapsed)
t.Errorf("bench with cancelled context took %v, want a prompt return", elapsed)
}
}
13 changes: 7 additions & 6 deletions docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ Declarative architecture docs (English, present tense). These describe the syste

- [overview.md](./overview.md) — purpose, consumers, components, provisioning order.
- [configuration.md](./configuration.md) — every setting changed from the PostgreSQL default, with why.
- [provisioning.md](./provisioning.md) — the tool's commands, idempotency rules, and the
reader-access chain.
- [provisioning.md](./provisioning.md) — the tool's two commands, idempotency rules, the
reader-access chain, and the stated assumption about the SCADA meeting an existing `trends`.

## Locked decisions

Expand All @@ -18,12 +18,13 @@ Declarative architecture docs (English, present tense). These describe the syste
| Installation | PostgreSQL via `winget`, major version pinned, automatic upgrade disabled; engine install stays outside the tool |
| Major version | 17 for new installs, pinned latest; floor 14 (SemiPlot needs `date_bin`; Simple-Scada 2 docs require 14+) |
| Provisioning tool | Go single binary `semibase.exe` (`pgx/v5`, embedded SQL); no runtime or `psql` on the target machine |
| Instance configuration | Fixed `ALTER SYSTEM` deltas applied by `semibase config` via `pg_reload_conf()`, never by editing `postgresql.conf`; `shared_buffers` takes effect at the next service restart, reported through `pg_settings.pending_restart` |
| Instance configuration | Fixed `ALTER SYSTEM` deltas applied by `semibase site` via `pg_reload_conf()`, never by editing `postgresql.conf`; `shared_buffers` takes effect at the next service restart, reported through `pg_settings.pending_restart` |
| Hardware floor | Installation machines guarantee 8 GB RAM and 50 GB database disk; memory settings are fixed constants sized to that floor, identical on every machine |
| Roles | `scada_writer` (SCADA), `semiplot_reader` (viewers, `SELECT` only); `semiplot_*` objects owned by `postgres` — a dedicated owner role appears when a tag-editing mechanism exists |
| Reader access | `ALTER DEFAULT PRIVILEGES FOR ROLE scada_writer` set **before** the writer first runs |
| Objects we add | `semiplot_tags` only — no triggers, functions, scheduled jobs, or extensions |
| Archive schema | Owned by Simple-Scada 2; documented in the SemiPlot repository, never created or altered here |
| Command surface | Two commands, `site` and `bench`, named for the situation rather than for the tool's internal steps; they differ in one thing, the memory tuning, which only `site` applies |
| Reader access | `ALTER DEFAULT PRIVILEGES FOR ROLE scada_writer` set **before** `public.trends` is created, and before the writer first runs |
| Objects we add | `semiplot_tags` and the vendor-shaped `public.trends` — no triggers, functions, scheduled jobs, or extensions; `messages` is not created, nothing we ship reads it |
| Archive schema | Owned by Simple-Scada 2 and documented in the SemiPlot repository. This tool creates `public.trends` once, with the vendor's shape, connected as `scada_writer`, and never alters it afterwards; day partitions and every later change to the schema remain the vendor's |
| Distribution | Binaries as GitHub release assets; the Linux binary also as `ghcr.io/semiteq/semibase`, `FROM scratch`, one file, pushed after the release exists; benches track `:latest` and a prerelease never moves it |
| Backup method | `UNDECIDED` — commissioning-time, needs the customer's archive size |
| Retention depth / disk sizing | `UNDECIDED` — needs a measured write rate from a working installation |
Expand Down
19 changes: 12 additions & 7 deletions docs/architecture/configuration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Instance configuration

Every setting the `config` command writes is listed. Most differ from the PostgreSQL default;
Every setting the `site` command writes is listed. Most differ from the PostgreSQL default;
two (`effective_cache_size`, `checkpoint_completion_target`) equal it and are pinned
deliberately, so the values in force are visible here rather than implied. The workload is a
sustained append-only insert stream from one writer plus a small number of read-heavy
Expand All @@ -10,12 +10,17 @@ Every installation machine guarantees at least 8 GB of RAM and 50 GB of disk for
so the memory figures are fixed constants sized to that floor: every machine runs the identical
configuration, and no setting depends on inspecting the host.

The `config` command of `semibase.exe` writes these through `ALTER SYSTEM`, so they live in
`postgresql.auto.conf` and survive a reinstall of `postgresql.conf`, and applies them with
`pg_reload_conf()`. Every setting below except `shared_buffers` is reload-context or weaker and
takes effect immediately; `shared_buffers` takes effect at the next service restart or reboot.
`config` prints the server's own pending-restart list (`pg_settings.pending_restart`), and
`verify` warns while any setting still waits, so a missed restart surfaces during commissioning.
`semibase site` writes these through `ALTER SYSTEM`, so they live in `postgresql.auto.conf` and
survive a reinstall of `postgresql.conf`, and applies them with `pg_reload_conf()`. Every setting
below except `shared_buffers` is reload-context or weaker and takes effect immediately;
`shared_buffers` takes effect at the next service restart or reboot. The tuning phase prints the
server's own pending-restart list (`pg_settings.pending_restart`), and the tail of the run reads it
again as its last statement, so a missed restart is the last thing the run says. `bench` reads it
too — it tunes nothing, but it runs against existing servers, and one that a `site` run tuned can
still be waiting for its restart.

`semibase bench` writes none of them: the values are constants sized to the installation
machine's hardware floor and mean nothing to a throwaway container.

## Server settings

Expand Down
Loading