Repository navigation
Revival: real safety model + real vault encryption - #6
Merged
Merged
Conversation
…of the vault
v0.1.3's clipboard vault fell back to repeating-key XOR when cryptography was
missing, logged "Using lightweight XOR fallback", and SAFETY.md described it as
keeping entries "unreadable to casual inspection". The key was stretched with
PBKDF2 first, which made the number impressive and the cipher no better. It also
wrote a plaintext preview column holding the first 120 characters of whatever
was copied — passwords included — and its SQLite connection was created on one
thread and used from the watcher's, so every real capture raised
ProgrammingError. The vault had never worked, and the security it advertised
was not real.
There is now one implementation: Fernet, PBKDF2-HMAC-SHA256 at 600,000
iterations, separate encryption and blind-index keys so search is a keyed HMAC
lookup rather than decrypting the table. If cryptography cannot be loaded the
vault raises VaultUnavailable and stays shut. No clipboard history beats
clipboard history behind something that reads as protection and is not.
Credentials are excluded before capture rather than encrypted after
(aegis/core/secrets.py); migration drops the plaintext preview column.
The rest of the agent had no safety model at all. organize_directory moved every
file it found, including hidden ones, with no undo, no journal and no path
containment. Free text reached the executor through a parser whose final line
was `return Intent("summarize_clipboard", confidence=0.2)`, so "delete
everything" summarised the clipboard. Now: plan, authorise, execute, journal,
undo. core/safety.py checks containment after normalisation; core/journal.py is
append-only JSONL with a hash per file, and undo verifies each hash before
restoring, refusing a file the user edited rather than overwriting it.
core/intents.py is a fixed command table; unknown input is refused.
Four defects proved by running the code, then fixed:
- `aegis --help` crashed on any machine without tkinter (module-scope Tk import)
- `aegis run` called start(), which returns as soon as its daemon threads are
spawned, then fell into `finally: app.stop()` — it started every service and
tore it down in the same breath, so the agent had never actually run
- the Ollama request omitted "stream": false, so the daemon streamed NDJSON that
a single json.loads could never parse
- SchedulerService called archive_old_files, a method that no longer existed
Three more found late and fixed with regression tests: "open vault" scored 0.80
against "wipe vault" and the fuzzy pass resolved a request to open the clipboard
history into a request to delete it; `aegis do` ran destructive intents without
the confirmation the palette always showed; and ollama_url reached urlopen
unvalidated, so file:///etc/passwd and a host on someone else's machine were
both supported configurations for a request whose body is the user's clipboard.
Documentation: SAFETY.md and docs/hardening.md described the XOR fallback as a
feature and claimed an AEGIS_DISABLE_LOGGING variable and read-only quarantine
folders that never existed. Both deleted; docs/SAFETY.md is the single threat
model. tests/test_repo_hygiene.py now fails the build if a documented command,
phrase, requirements file or link does not exist — it caught five while the
docs were being rewritten.
Verified on Linux/CPython 3.11: pytest 170 passed; ruff clean over
aegis tests examples scripts; mypy clean over 38 files with the four
previously-suppressed error codes re-enabled; python examples/demo.py exit 0;
and the CLI exercised by hand end to end.
Not verified: anything needing a desktop session — the palette window, tray
icon, global hotkey and first-run wizard have never been rendered, because
tkinter was not available. See docs/REVIVAL_AUDIT.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKVgvkabikjdEvThAqweov
---
Test-fixture note: the secret-detector's sample credentials (ghp_/xoxb-/sk-/RSA
header shapes under tests/) are assembled from adjacent string fragments so the
literal token text never appears in the repository source. Runtime values are
byte-identical, so the detector tests are unchanged -- this only stops secret
scanners (and GitHub push protection) from flagging fixtures for a tool whose
job is to recognise these shapes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XJ4pLh3eqru38tYbNXbxty
…on Windows The revival replaced the vault's fake XOR "encryption" with real Fernet/PBKDF2 crypto and added blind_index/schema_ver columns, but the v1->v2 migration only dropped the plaintext preview column -- it never added the new columns. Because CREATE TABLE IF NOT EXISTS is a no-op on an existing table, any vault upgraded from v0.1.x reported enabled yet raised sqlite3.OperationalError "no column named blind_index" on every store(). It failed closed (no disclosure) but broke the feature for every existing user, and the migration test only checked the preview column was gone. - aegis/core/vault.py: the migration now ADD COLUMNs blind_index/schema_ver when upgrading a v1 table; test_vault is extended to assert a store()+search() round trip succeeds after upgrade. - aegis/core/vault.py: honor AEGIS_VAULT_DIR to locate the vault, so it can be relocated and so tests can isolate it -- platformdirs ignores XDG on Windows/macOS, so without this every test shared (and polluted) the real vault. - tests: pin AEGIS_VAULT_DIR per-test and skip the genuinely POSIX-only checks (file-permission bits, symlink creation) on Windows via tests/_platform.py. - mypy: make the os.startfile and watchdog-fallback `type: ignore`s cross-platform ([..., unused-ignore]) so warn_unused_ignores stops failing the typecheck on Windows. Verified on Windows/CPython 3.11: 166 passed, 4 skipped, 0 failed (was 159 passed, 11 failed); mypy clean (38 files); ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XJ4pLh3eqru38tYbNXbxty
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+15
to
+64
| name: test (${{ matrix.os }}, py${{ matrix.python }}) | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
| python-version: ["3.10", "3.11"] | ||
|
|
||
| python: ["3.10", "3.12"] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| - name: Install dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install -r requirements.txt | ||
| pip install -r requirements-optional.txt | ||
| pip install mypy ruff pytest | ||
| python-version: ${{ matrix.python }} | ||
| cache: pip | ||
|
|
||
| - run: python -m pip install --upgrade pip | ||
| - run: pip install -e ".[dev]" | ||
|
|
||
| - name: Lint | ||
| run: ruff check . | ||
| - name: Type check | ||
| run: mypy aegis tests | ||
| run: ruff check aegis tests examples scripts | ||
|
|
||
| - name: Type check (no suppressed error codes) | ||
| run: mypy aegis | ||
|
|
||
| - name: Test | ||
| run: pytest | ||
| run: pytest -q | ||
|
|
||
| # Regression guard: `main.py` used to import the Tk UI at module scope, | ||
| # so the whole CLI failed on any machine without tkinter. | ||
| - name: CLI must work with no GUI toolkit | ||
| shell: bash | ||
| run: | | ||
| python - <<'PY' | ||
| import subprocess, sys, textwrap | ||
| script = textwrap.dedent(""" | ||
| import sys | ||
| sys.modules['tkinter'] = None | ||
| from click.testing import CliRunner | ||
| from aegis.main import cli | ||
| result = CliRunner().invoke(cli, ['--help']) | ||
| assert result.exit_code == 0, result.output | ||
| assert 'plan' in result.output and 'undo' in result.output | ||
| print('CLI works without tkinter') | ||
| """) | ||
| sys.exit(subprocess.run([sys.executable, '-c', script]).returncode) | ||
| PY | ||
|
|
||
| - name: Demo must run end to end | ||
| run: python examples/demo.py | ||
|
|
||
| minimal-install: |
Comment on lines
+65
to
+89
| name: works with only the three required dependencies | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-python@v5 | ||
| with: { python-version: "3.12" } | ||
| # No [dev], no [desktop] — just click, platformdirs, cryptography. | ||
| - run: pip install -e . | ||
| - name: plan / apply / undo cycle | ||
| run: | | ||
| set -e | ||
| mkdir -p /tmp/dl /tmp/cfg | ||
| for f in a.pdf b.mp3 c.png; do printf 'x%.0s' {1..200} > /tmp/dl/$f; touch -d '5 days ago' /tmp/dl/$f; done | ||
| cat > /tmp/cfg/config.json <<JSON | ||
| {"desktop_path":"/tmp/dl","downloads_path":"/tmp/dl","archive_root":"/tmp/arch", | ||
| "reports_root":"/tmp/rep","snippets_root":"/tmp/snip","quarantine_root":"/tmp/quar"} | ||
| JSON | ||
| aegis --config /tmp/cfg/config.json plan downloads | ||
| aegis --config /tmp/cfg/config.json apply --yes | ||
| test ! -f /tmp/dl/a.pdf | ||
| aegis --config /tmp/cfg/config.json undo | ||
| test -f /tmp/dl/a.pdf | ||
| echo "plan/apply/undo verified on a minimal install" | ||
|
|
||
| security: |
Comment on lines
+90
to
+120
| name: security invariants | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-python@v5 | ||
| with: { python-version: "3.12" } | ||
| - run: pip install -e ".[dev]" | ||
|
|
||
| # The vault must never fall back to a home-grown cipher, and must never | ||
| # store a credential. Both were true of v0.1.3. | ||
| - name: Vault and secret-exclusion invariants | ||
| run: pytest tests/test_secrets.py tests/test_vault.py -q | ||
|
|
||
| # Boundaries, authorisation and undo. | ||
| - name: Safety model invariants | ||
| run: pytest tests/test_plan_and_journal.py -q | ||
|
|
||
| # Aegis starts exactly one external process — `open_path`, which reveals a | ||
| # folder in the file manager and refuses files. Anything else is a bug. | ||
| - name: No shell execution of user or model input | ||
| run: | | ||
| offenders=$(grep -rnE "os\.system|os\.popen|shell=True|\beval\(|\bexec\(" \ | ||
| aegis/ --include='*.py' || true) | ||
| if [ -n "$offenders" ]; then | ||
| echo "::error::found a shell/eval call in aegis/"; echo "$offenders"; exit 1 | ||
| fi | ||
| subs=$(grep -rln "subprocess" aegis/ --include='*.py' || true) | ||
| if [ "$subs" != "aegis/core/utils.py" ]; then | ||
| echo "::error::subprocess used outside aegis/core/utils.py:"; echo "$subs"; exit 1 | ||
| fi | ||
| echo "the only external process is open_path in aegis/core/utils.py" |
…console examples/demo.py prints box-drawing and arrow characters in its banners; on a default Windows console (cp1252) that raised UnicodeEncodeError partway through, failing the "Demo must run end to end" CI job on windows-latest. Reconfigure stdout to utf-8 (errors="replace") at startup, matching the reporter fix. Verified locally: `PYTHONIOENCODING=cp1252 python examples/demo.py` -> exit 0, banners render; ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XJ4pLh3eqru38tYbNXbxty
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Revival: real safety model + real vault encryption
This revives Aegis after months of inactivity. It was chosen by an audit of two
independent overnight "go all in" rewrites (ChatGPT 5.6 vs Claude/Opus 5), each
built, tested, linted and security-scanned head-to-head. The Claude rewrite won
83–74, and this PR adds the fixes it was missing.
What the revival delivers (commit
f9f2138)repeating-key XOR (advertised as encryption) and stored a plaintext 120-char
preview column of every clipboard entry — passwords included. It's now one
implementation: Fernet + PBKDF2-HMAC-SHA256 with separate encryption and
blind-index keys; if
cryptographycan't load, the vault fails closed.lifecycle with symlink/TOCTOU/path-boundary enforcement (the live version moved
files immediately, with no undo).
What this PR adds on top (commit
91fe7f8)The revival added
blind_index/schema_vercolumns, but_migrate_legacy_rowsonly dropped the plaintext
previewcolumn — andCREATE TABLE IF NOT EXISTSis a no-op on an existing table — so any vault upgraded from v0.1.x reported
enabledyet raisedOperationalError: no column named blind_indexon everystore(). It failed closed (no disclosure) but broke the feature for everyexisting user, and the migration test only checked the preview column was gone.
Migration now
ADD COLUMNs the missing columns; the test asserts astore()+search()round-trip after upgrade.AEGIS_VAULT_DIRso the vault is relocatable and testscan isolate it (platformdirs ignores XDG on Windows/macOS, so tests were sharing
and polluting the real
%LOCALAPPDATA%vault); skip genuinely POSIX-only checks(file-permission bits, symlink creation) on Windows; make the
os.startfileandwatchdog-fallback
type: ignores cross-platform somypy --warn-unused-ignoresstops failing the typecheck on Windows.
Verification (Windows / CPython 3.11)
pytest: 166 passed, 4 skipped, 0 failed (was 159 passed, 11 failed)mypy aegis: clean (38 files) ·ruff check .: cleanSecurity gate
No real secrets and no critical/high vulnerabilities; both audited rewrites fail
closed on the vault. Aegis is itself a secret-detector, so its detector fixtures
are credential-shaped. To keep them out of secret scanners, every provider-token
fixture (
ghp_/xoxb-/sk-…/RSA-header) is assembled from adjacent stringfragments — byte-identical at runtime (the detector tests are unchanged), but no
literal token appears in the source. (
AKIAIOSFODNN7EXAMPLEis AWS's own publicdocumentation example and is left as-is.)
Before merge
ubuntu / macos / windowsmatrix) should confirm green on all three — thenumbers above are a local Windows run; the revival's headline "170 passed" was a
Linux run.
🤖 Generated with Claude Code