diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 06d8bb1..0d95f07 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,8 +23,8 @@ jobs: runs-on: ${{ matrix.os }} timeout-minutes: 10 steps: - - uses: actions/checkout@v7 - - uses: astral-sh/setup-uv@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0 with: python-version: ${{ matrix.python-version }} enable-cache: true @@ -39,8 +39,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v7 - - uses: astral-sh/setup-uv@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0 with: python-version: "3.12" enable-cache: true diff --git a/CHANGELOG.md b/CHANGELOG.md index f66beda..ecd7057 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,24 +1,32 @@ # Changelog -## 0.6.0 +## 0.6.1 - 2026-10-04 -Publication-ready portfolio release. +### Fixed -### Added +- runtime version reporting now follows the package version; +- Unicode letters are preserved during text normalization; +- Excel datetime values participate in date matching; +- common US and European amount formats, including accounting-style negatives, are parsed consistently; +- optional matching rules keep their documented base weights before active weights are normalized. -- polished project documentation and architecture notes; -- security policy and contribution guide; -- repository issue and pull-request templates; -- dependency update configuration; -- package build verification in CI; -- curated product screenshots. +### Changed -### Included from earlier milestones +- matching documentation now describes the score-ordered one-to-one selection behavior precisely; +- public documentation and UI copy were simplified and made consistent. + +## 0.6.0 + +Initial public release. + +### Added -- CSV/XLSX ingestion and preview; -- bounded candidate generation and deterministic scoring; +- CSV and XLSX ingestion with preview and validation; +- candidate generation and typed matching rules; - one-to-one reconciliation with ambiguity review; -- human accept/reject/manual-link workflow and audit trail; +- human accept, reject, and manual-link actions with audit history; - SQLite persistence, pagination, and large-run safeguards; -- CSV/XLSX exports with formula-injection mitigation; -- local web hardening and staged-file cleanup. +- CSV and XLSX exports with formula-injection mitigation; +- local web protections and staged-file cleanup; +- architecture, security, and contribution documentation; +- CI package build verification and dependency update configuration. diff --git a/README.md b/README.md index ebe65d2..7d5344b 100644 --- a/README.md +++ b/README.md @@ -2,68 +2,26 @@ [![CI](https://github.com/SerDrozd/rowbridge/actions/workflows/ci.yml/badge.svg)](https://github.com/SerDrozd/rowbridge/actions/workflows/ci.yml) -**Local-first reconciliation for messy CSV and Excel exports.** +**Local-first reconciliation for CSV and Excel exports.** -RowBridge helps you compare two exports when the same real-world record is represented differently on each side. It combines deterministic matching, bounded candidate generation, explainable evidence, human review, and auditable exports without sending source data to an external service. +RowBridge compares two tabular exports, proposes one-to-one matches, sends uncertain pairs to human review, and exports the current result with an audit trail. Source data stays on the machine running the app. ![RowBridge results](docs/images/results.png) -## Why it exists - -Operational reconciliation often starts with two spreadsheets and no reliable shared key. One file may contain `INV-00123`, while another contains `INV00123`; names vary, dates drift by a day, and numeric values differ by a few cents. - -A spreadsheet lookup works until it does not. RowBridge is designed for the cases where you need to answer three questions clearly: - -1. Which rows are probably the same record? -2. Why did the matcher pair them? -3. Which decisions were made by a human afterwards? - -## What RowBridge does - -- imports CSV and XLSX files locally; -- detects common CSV delimiters and UTF-8 / UTF-16 / Windows-1252 encodings; -- previews both sources before matching; -- maps primary, secondary text, amount, and date fields; -- generates a bounded candidate set instead of scoring the full Cartesian product; -- uses deterministic fuzzy text, numeric tolerance, and date-window rules; -- preserves one-to-one matching; -- sends ambiguous candidates to a review queue; -- supports accept, reject, and manual-link decisions; -- persists runs and review history in SQLite; -- paginates large result sets; -- exports the full current state as CSV or an XLSX workbook; -- keeps source processing local to the machine running RowBridge. - -## Workflow - -### 1. Load two exports - -![RowBridge file selection](docs/images/home.png) - -### 2. Preview and map equivalent fields - -![RowBridge field mapping](docs/images/mapping.png) - -### 3. Review uncertain matches - -Each proposed match shows the evidence that contributed to its score. Review actions are stored separately from the original algorithmic evidence. - -### 4. Export the result - -CSV exports include both source payloads and matching evidence. Workbook exports include summary, reconciliation, unmatched rows, and review history sheets. - -## Quick start +## Get started Requirements: Python 3.12+ and [`uv`](https://docs.astral.sh/uv/). ```bash +git clone https://github.com/SerDrozd/rowbridge.git +cd rowbridge uv sync --extra dev uv run rowbridge ``` -Then open `http://127.0.0.1:8000`. +Open `http://127.0.0.1:8000`. -For the included demo: +The included demo files can be reconciled with these mappings: | Role | Side A | Side B | | --- | --- | --- | @@ -73,13 +31,24 @@ For the included demo: | Amount | `amount` | `total` | | Date | `invoice_date` | `paid_at` | -Application state is written to `.rowbridge/` by default. Set `ROWBRIDGE_DATA_DIR` to use another directory. +Application state is stored in `.rowbridge/` by default. Set `ROWBRIDGE_DATA_DIR` to use another directory. + +## Typical workflow + +1. Load one CSV or XLSX export on each side. +2. Preview the files and map equivalent fields. +3. Review uncertain matches and accept, reject, or manually link rows. +4. Export the current reconciliation as CSV or XLSX. + +![RowBridge file selection](docs/images/home.png) + +![RowBridge field mapping](docs/images/mapping.png) -## Matching model +## How matching works -RowBridge does not treat fuzzy matching as a black box. A match is built from typed comparison rules. +A run always has a primary identity field. Secondary text, amount, and date fields are optional. -With all four rule types enabled, the default weights are: +With all four signals enabled, the base weights are: | Signal | Weight | | --- | ---: | @@ -88,71 +57,43 @@ With all four rule types enabled, the default weights are: | Amount | 15% | | Date | 10% | -If optional rules are omitted, the remaining weights are normalized. - -Candidate generation uses exact normalized primary blocks first, then supporting blocks. Only rows that still lack candidates can enter a bounded fuzzy fallback. On large inputs, global fuzzy fallback is disabled rather than silently turning into an all-against-all scan. - -A selected pair is sent to review when its score is below the automatic-match threshold or when a close competitor exists for either source row. +If optional fields are not mapped, their weights are removed and the remaining active weights are normalized. -More detail: [`docs/MATCHING.md`](docs/MATCHING.md). +Candidate generation checks normalized primary values first, then supporting blocks. Rows that still have no candidate can use a limited fuzzy fallback when the target side is small enough to scan. -## Architecture - -```text -CSV / XLSX - | - v -Ingestion + validation - | - v -Candidate generation ----> typed comparison rules - | | - +------------+--------------+ - v - one-to-one resolver - | - v - SQLite run state - | - +--------+--------+ - | | - v v - human review exports -``` +Scored candidates are sorted by score, then by source row indexes. RowBridge walks that order and selects the first non-conflicting pairs. This is deterministic score-ordered selection, not an optimization for maximum total score. -The application is intentionally a single local FastAPI process with server-rendered HTML and SQLite. There is no cloud database, frontend build chain, or required external API. +A selected pair remains in review when its score is below the automatic-match threshold or when a close competing candidate exists for either source row. -More detail: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). +More detail is in [`docs/MATCHING.md`](docs/MATCHING.md). -## Input handling +## Inputs and limits -CSV input supports comma, semicolon, tab, and pipe delimiters. Encodings currently supported are UTF-8 with or without BOM, UTF-16, and Windows-1252. +CSV input supports comma, semicolon, tab, and pipe delimiters. Supported encodings are UTF-8 with or without BOM, UTF-16, and Windows-1252. -XLSX files are opened read-only. RowBridge chooses the first worksheet containing a non-empty header and at least one data row. +XLSX files are opened read-only. RowBridge uses the first worksheet that contains a non-empty header and at least one data row. -Current limits per source file: +Each source file is limited to: - 5 MB upload size; - 50,000 data rows; - 200 columns. -Duplicate headers, empty headers, malformed rows, oversized workbook archives, unsupported file types, and obviously invalid amount/date field mappings are rejected with an explicit error. +Duplicate headers, empty headers, malformed rows, oversized workbook archives, unsupported file types, and invalid amount/date mappings are rejected before a run is created. -## Human review and audit trail +## Review and audit trail -A review candidate can be: +A review candidate can be accepted or rejected. Unmatched rows can also be linked manually. -- **accepted** — retains the original score and evidence, but becomes human-confirmed; -- **rejected** — returns both source rows to the unmatched pool; -- **replaced with a manual link** — records a human decision without inventing an algorithmic confidence score. +Accepted pairs keep their algorithmic score and evidence. Rejected pairs return both rows to the unmatched pool. Manual links are stored as human decisions without inventing an algorithmic confidence score. -Every human mutation is appended to the review history with affected rows, timestamp, previous state, resulting state, and a short explanation. +Review actions are appended to the run history with the affected rows, timestamp, previous state, resulting state, and a short explanation. ## Exports -The CSV export contains the current status, score, source row numbers, source payloads, and matching evidence. +CSV exports contain the current status, score, source row numbers, source payloads, and matching evidence. -The workbook export contains: +XLSX exports contain: - `Summary`; - `Reconciliation`; @@ -162,27 +103,21 @@ The workbook export contains: Values that begin with spreadsheet formula prefixes are escaped before export to reduce formula-injection risk. -## Local-first and security boundaries +## Local data and security -RowBridge binds to `127.0.0.1` by default and the matching workflow has no external API dependency. Source files are treated as read-only inputs; temporary staged copies are deleted after successful run creation and stale stages expire automatically. +RowBridge binds to `127.0.0.1` by default and does not require an external matching API. Uploaded source files are used as local read-only inputs. Temporary staged copies are removed after successful run creation, and abandoned stages expire automatically. -The web layer uses trusted-host checks, restrictive response headers, `Cache-Control: no-store`, and browser Fetch Metadata to reject cross-site mutation requests when the browser supplies that signal. +The web layer applies trusted-host checks, restrictive response headers, `Cache-Control: no-store` for application pages, and browser Fetch Metadata checks for cross-site mutation requests. -This is a local single-user tool, not an authenticated multi-user web service. Do not expose the development server directly to an untrusted network. +RowBridge is a local single-user tool. Do not expose the development server directly to an untrusted network. See [`SECURITY.md`](SECURITY.md) for the supported security model. -## Performance choices +## Performance safeguards -- exact normalized identifiers take the shortest matching path; -- candidate sets are capped per source row; -- global fuzzy fallback is bounded; -- ambiguity lookup uses per-row competitor indexes; -- result pages are limited to 100 rows; -- large unmatched sets use explicit source-row inputs instead of thousands of `