Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "synalog"
version = "2.0.0"
version = "2.1.0"
edition = "2024"
description = "Logic programming for AI agents: Datalog-family language compiling to optimized SQL"
license = "Apache-2.0"
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,13 @@ $ synalog program.l run EngineeringTeam
- `--limit` / `--offset` paginate the result.
- `--csv` (with `run`) prints results as CSV instead of the rendered table.
- `--search REGEX` (with `print`/`run`) keeps only rows where some column matches the regular expression `REGEX`, e.g. `synalog program.l run Customers --search "(?i)acme"`. In the interactive session the same is `.search Customers (?i)acme`.
- `run` executes on `duckdb` (the default), `sqlite` (stdlib) or `psql` (with `--dsn` or `SYNALOG_PSQL_DSN`) out of the box: synalog depends on duckdb and on psycopg with its bundled libpq, on macOS and Linux alike. For other engines, install their driver (`pip install 'synalog[trino]'`, ...) or use `print` and run the SQL with your own client.
- `run` executes on `duckdb` (the default), `sqlite` (stdlib) or `psql` (in a [project](#projects-layertoml) connected to it) out of the box: synalog depends on duckdb and on psycopg with its bundled libpq, on macOS and Linux alike. For other engines, install their driver (`pip install 'synalog[trino]'`, ...) or use `print` and run the SQL with your own client.
- `import path.to.file.Pred;` statements resolve `path/to/file.l` against the program file's directory, then the current directory; pass `--import-root DIR` (repeatable) to search elsewhere.
- `--load TABLE=PATH` (repeatable) loads a csv/tsv/json/jsonl/parquet file as a table before running, e.g. `synalog senior.l run Senior --load employees=employees.csv`.

### Projects: `synalog.toml`
### Projects: `layer.toml`

A folder with a `synalog.toml` is a project. Its `[connection]` says which database its programs run on, as plain fields — commit it:
A folder with a `layer.toml` is a project. Its `[connection]` says which database its programs run on, as plain fields — commit it:

```toml
[connection]
Expand All @@ -138,7 +138,7 @@ user = "analyst"
schema = "public"
```

Secrets never go in the file (synalog refuses it): each comes from the environment as `SYNALOG_<ENGINE>_<FIELD>` — `SYNALOG_PSQL_PASSWORD`, `SYNALOG_DATABRICKS_ACCESS_TOKEN`, or `GOOGLE_APPLICATION_CREDENTIALS` for BigQuery — usually from the project's `.env`, kept out of git. Inside the project (from the program's folder or the current directory, and their parents), `run`, `print` and `verify` use that engine and connection, imports resolve from the project's folder, and `synalog introspect` writes the project's `tables/`, one file per table of the database; `--engine`, an `@Engine` annotation, `--dsn` and `SYNALOG_<ENGINE>_DSN` still take precedence. The fields of every engine are in `synalog.project.ENGINES`.
Secrets never go in the file (synalog refuses it): each comes from the environment as `SYNALOG_<ENGINE>_<FIELD>` — `SYNALOG_PSQL_PASSWORD`, `SYNALOG_DATABRICKS_ACCESS_TOKEN`, or `GOOGLE_APPLICATION_CREDENTIALS` for BigQuery — usually from the project's `.env`, kept out of git. Inside the project (from the program's folder or the current directory, and their parents), `run`, `print` and `verify` use that engine and connection, imports resolve from the project's folder, and `synalog introspect` writes the project's `tables/`, one file per table of the database. The project's file is the only place a connection comes from; `--engine` and an `@Engine` annotation choose the dialect, and a remote engine runs only through the project's connection to it. `synalog connect <engine> key=value ...` writes it — the non-secret fields to `layer.toml`, the secret ones to `.env` — `synalog connect` shows it (secrets hidden), `synalog connect clear` removes it. The fields of every engine are in `synalog.project.ENGINES`.

Running `synalog` with no arguments starts an interactive session, in the spirit of `python` (the options above, e.g. `--engine` or `--load`, apply to it too):

Expand Down Expand Up @@ -220,17 +220,17 @@ for name, sql in sqls.items():
print(name, sql)
```

### `execute(source, predicate, engine=None, dsn=None, import_root=None, limit=None, offset=None, pattern=None, assertion=None, loads=()) -> tuple[list[str], list[tuple]]`
### `execute(source, predicate, engine=None, project=None, import_root=None, limit=None, offset=None, pattern=None, assertion=None, loads=()) -> tuple[list[str], list[tuple]]`

Run a predicate on its database (the program's or project's engine and connection) and return `(columns, rows)`. Each recursion stops as soon as a step changes nothing, so it costs the steps its data needs, and `@Recursive(P, -1)` (until nothing changes) runs on every engine. `plan()` returns the same steps for a host that runs them itself.

```python
columns, rows = synalog.execute(source, "TopCustomers", limit=20)
```

### `check(source, engine=None, import_root=None, assertions=True, dsn=None) -> tuple[list[str], list[str]]`
### `check(source, engine=None, import_root=None, assertions=True, project=None) -> tuple[list[str], list[str]]`

Run structural validation. Returns the error messages and the warning messages, as two lists. The program is valid when there is no error; warnings do not make it invalid. Inside a project whose `synalog.toml` has a `[connection]` (or given a `dsn`), it also runs the program's `@Assert` statements against the database and reports each violated one as an error; `assertions=False` keeps it offline.
Run structural validation. Returns the error messages and the warning messages, as two lists. The program is valid when there is no error; warnings do not make it invalid. Inside a project whose `layer.toml` has a `[connection]` — the current directory's, or the folder `project` names — it also runs the program's `@Assert` statements against the database and reports each violated one as an error; `assertions=False` keeps it offline.

```python
errors, warnings = synalog.check(source)
Expand All @@ -243,15 +243,15 @@ All of these functions accept an optional `engine` keyword that overrides the pr

## Language overview

A Synalog program is a **project**: a folder with a `synalog.toml`, holding one predicate per file in three folders, the layout of [semantic layers](https://github.com/SynaLinks/semantic-layers):
A Synalog program is a **project**: a folder with a `layer.toml`, holding one predicate per file in three folders, the layout of [semantic layers](https://github.com/SynaLinks/semantic-layers):

```
shop/
├── tables/Orders.l # the data: one file per database table
├── concepts/Customer.l # entities and relationships extracted from tables
├── rules/CustomerRevenue.l # insights derived from concepts
├── rules/TopCustomers.l
└── synalog.toml # the project's name, description and database
└── layer.toml # the project's name, description and database
```

Each file is named after its predicate and opens with front matter (`name`, the predicate; `description`, what its rows are), then imports what it builds on by folder, file and predicate, then orders its predicate with `@OrderBy`. A database table is referenced by its lowercase database name in `tables/`, and everything else builds on the PascalCase predicate that maps it:
Expand Down
2 changes: 1 addition & 1 deletion docs/assertions.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ An assertion is checked by looking for its counterexamples: Synalog compiles tha
| `synalog program.l verify [Predicate ...]` | Runs every assertion, or those of the given predicates, and prints the counterexamples of each violated one (5 by default, `--limit` to change). Exits 1 if one is violated. |
| `synalog program.l run Predicate` | Checks the program's assertions before it prints anything, and exits 1 with a few counterexamples if one is violated. |
| `synalog program.l print Predicate` | Never touches the database: assertions are not run. |
| [`check()`](python-api.md#check) | Runs the assertions when a database is known: inside a project whose `synalog.toml` has a `[connection]`, or given a `dsn`. Each violated assertion is an error. `assertions=False` keeps it offline. |
| [`check()`](python-api.md#check) | Runs the assertions when a database is known: inside a project whose `layer.toml` has a `[connection]` (the current directory's, or the folder `project` names). Each violated assertion is an error. `assertions=False` keeps it offline. |

```text
$ synalog family.l run Grandparent --load parents=parents.csv
Expand Down
32 changes: 23 additions & 9 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,12 @@ $ synalog program.l run EngineeringTeam
| Option | Meaning |
| --- | --- |
| `-c PROGRAM` | Pass the program text inline instead of `FILE`, like `python -c`. |
| `--engine <name>` | Target SQL dialect. Resolution order: this flag, then the program's `@Engine` annotation, then `duckdb`. |
| `--engine <name>` | Target SQL dialect. Resolution order: this flag, then the program's `@Engine` annotation, then the project's engine, then `duckdb`. |
| `--limit N` / `--offset N` | Paginate the result. |
| `--csv` | With `run`: print results as CSV instead of a rendered table. |
| `--search REGEX` | With `print`/`run`: keep only rows where some column matches the regular expression `REGEX` (engine-native regex, not a SQL `LIKE` pattern). Applies to the filtered rows before pagination. |
| `--import-root DIR` | Directory where `import` statements look up `.l` files (repeatable). |
| `--load TABLE=PATH` | Load a csv/tsv/json/jsonl/parquet file as a table before running (repeatable). |
| `--dsn <conninfo>` | PostgreSQL connection string for `--engine psql` (or set `SYNALOG_PSQL_DSN`). |

Passing `-` as the file reads the program from stdin, and `-c` takes the program text directly, and both compose with the other options:

Expand All @@ -79,7 +78,7 @@ With `-c` there is no `FILE` argument: the positionals are the command and its p

- **duckdb**: the default engine, bundled with synalog; nothing extra to install.
- **sqlite**: Python's stdlib driver; Logica's runtime UDFs (ArgMin/ArgMax, ARRAY_CONCAT, ...) are registered when the `logica` package is installed.
- **psql**: bundled with synalog (psycopg with its own libpq); needs a connection string.
- **psql**: bundled with synalog (psycopg with its own libpq); runs in a [project](#projects-layertoml) connected to PostgreSQL.

The [`Today` and `Now`](language/temporal.md) built-in concepts need no runner support, since the compiler inlines them per dialect, so they work on every engine.

Expand All @@ -100,7 +99,7 @@ duckdb and the PostgreSQL driver ship with synalog. For the other engines (`bigq

### Imports

In a project, `import <folder>.<Name>.<Name>;` statements resolve from the project's folder (the one holding `synalog.toml`), whichever file runs and wherever from: `import tables.Orders.Orders;` reads the project's `tables/Orders.l`. The program file's directory and the current directory come next. Pass `--import-root DIR` (repeatable) to search elsewhere; explicit roots replace the defaults.
In a project, `import <folder>.<Name>.<Name>;` statements resolve from the project's folder (the one holding `layer.toml`), whichever file runs and wherever from: `import tables.Orders.Orders;` reads the project's `tables/Orders.l`. The program file's directory and the current directory come next. Pass `--import-root DIR` (repeatable) to search elsewhere; explicit roots replace the defaults.

In the [`shop` project](language/index.md), `rules/TopCustomers.l` builds on `rules/CustomerRevenue.l`:

Expand Down Expand Up @@ -155,9 +154,9 @@ Compile error: No rules are defining 'Missing', but compilation was requested.

A failing program never produces partial output: `run` either prints the table or the error.

## Projects: `synalog.toml`
## Projects: `layer.toml`

A Synalog program is a [project](language/index.md): a folder with a `synalog.toml`, its predicates in `tables/`, `concepts/` and `rules/`, one per file. The file names and describes the project, and its `[connection]` says which database it runs on, as plain fields — commit it:
A Synalog program is a [project](language/index.md): a folder with a `layer.toml`, its predicates in `tables/`, `concepts/` and `rules/`, one per file. The file names and describes the project, and its `[connection]` says which database it runs on, as plain fields — commit it:

```toml
[project]
Expand All @@ -173,7 +172,22 @@ user = "analyst"
schema = "public"
```

Secrets never go in the file (synalog refuses it): each comes from the environment as `SYNALOG_<ENGINE>_<FIELD>` — `SYNALOG_PSQL_PASSWORD`, `SYNALOG_DATABRICKS_ACCESS_TOKEN`, or `GOOGLE_APPLICATION_CREDENTIALS` for BigQuery — usually from the project's `.env`, kept out of git. Anywhere in the project, `run`, `print`, `verify` and `introspect` use that engine and connection, and [imports](#imports) resolve from the project's folder; `--engine`, an `@Engine` annotation, `--dsn` and `SYNALOG_<ENGINE>_DSN` still take precedence. The fields of every engine are in `synalog.project.ENGINES`.
Secrets never go in the file (synalog refuses it): each comes from the environment as `SYNALOG_<ENGINE>_<FIELD>` — `SYNALOG_PSQL_PASSWORD`, `SYNALOG_DATABRICKS_ACCESS_TOKEN`, or `GOOGLE_APPLICATION_CREDENTIALS` for BigQuery — usually from the project's `.env`, kept out of git. Anywhere in the project, `run`, `print`, `verify` and `introspect` use that engine and connection, and [imports](#imports) resolve from the project's folder. The project's file is the only place a connection comes from: `--engine` and an `@Engine` annotation choose the dialect, and a remote engine runs only through the project's connection to it. The fields of every engine are in `synalog.project.ENGINES`.

### Connect

```console
$ synalog connect psql host=db.example.com database=sales user=analyst password=...
Connected /path/to/sales/layer.toml to psql (secrets in .env)
$ synalog connect
/path/to/sales/layer.toml
engine = psql
host = db.example.com
...
password: SYNALOG_PSQL_PASSWORD set
```

`synalog connect <engine> key=value ...` writes it — the non-secret fields to `layer.toml`, the secret ones to `.env` — `synalog connect` shows it (secrets hidden), `synalog connect clear` removes it. It runs in the project's folder — the nearest `layer.toml`'s, else the current directory — keeps the file's other tables, adds `.env` to `.gitignore`, and writes nothing when a required field is missing.

### Introspect

Expand All @@ -193,7 +207,7 @@ description: Orders.
PublicOrders(order_id:, customer_id:, amount:, status:) :- public.orders(order_id:, customer_id:, amount:, status:);
```

Run it again when the schema changes: the declarations are regenerated, and a file's front matter (its description, keywords, ...) is kept as written. A table the database no longer has keeps its file, since concepts may import it, and is listed. Outside a project, `synalog introspect <engine> [dsn]` prints the declarations instead. PostgreSQL, Trino, Presto, Databricks and BigQuery can be introspected.
Run it again when the schema changes: the declarations are regenerated, and a file's front matter (its description, keywords, ...) is kept as written. A table the database no longer has keeps its file, since concepts may import it, and is listed. It needs a project connected to its database. PostgreSQL, Trino, Presto, Databricks and BigQuery can be introspected.

## Add the skill to your coding agent

Expand Down Expand Up @@ -243,7 +257,7 @@ Bad(x) :- x ==<EMPTY>
Greeting("hi");
```

The `--engine`, `--dsn`, `--import-root` and `--load` options also apply to the session:
The `--engine`, `--import-root` and `--load` options also apply to the session, and the project's connection too:

```bash
synalog --engine sqlite --load employees=employees.csv
Expand Down
2 changes: 1 addition & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ It deploys to GitHub Pages automatically on push via `.github/workflows/docs.yml

### Documentation examples

The example projects (`docs/examples/shop/`, `docs/examples/smoke/`) are laid out as every Synalog project is: `tables/`, `concepts/`, `rules/`, a `synalog.toml`, and their data in `data/<table>.csv`. `run.py` loads the data into an in-memory DuckDB, checks every file, runs the predicate its front matter names, and writes the results to `<project>.log`.
The example projects (`docs/examples/shop/`, `docs/examples/smoke/`) are laid out as every Synalog project is: `tables/`, `concepts/`, `rules/`, a `layer.toml`, and their data in `data/<table>.csv`. `run.py` loads the data into an in-memory DuckDB, checks every file, runs the predicate its front matter names, and writes the results to `<project>.log`.

The feature examples (aggregation, functors, recursion, ...) are single `.l` programs with their data as inline facts. For those, the `run.py` harness validates each program with `synalog.check()`, compiles every predicate listed in its `# run:` header, executes the SQL on an in-memory DuckDB, and writes the combined output to the matching `.log` file. A `# load: <file>.csv as <table>` header loads a CSV from `docs/examples/` into a DuckDB table before the predicates run. The `Today`/`Now` built-in concepts need no special handling, since the compiler inlines them per dialect:

Expand Down
Loading
Loading