Skip to content

Make Photos PhotoKit-first - #107

Merged
jmcte merged 3 commits into
mainfrom
codex/issue-94-photokit-photos
Jul 28, 2026
Merged

Make Photos PhotoKit-first#107
jmcte merged 3 commits into
mainfrom
codex/issue-94-photokit-photos

Conversation

@jmcte

@jmcte jmcte commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • use bounded PhotoKit reads as the primary Photos logical-asset source
  • return identifiers, media facts, dimensions, favorite/hidden state, album membership, and explicit provenance
  • avoid pixels, thumbnails, network requests, and implicit iCloud downloads
  • retain the existing package walk only as --degraded-filesystem

Governing Issue

Closes #94

Validation

  • Relevant local checks passed

  • Required PR checks are expected to satisfy CI Gate

  • Skipped checks are explained below

  • swift test --disable-sandbox (117 tests passed)

  • bash scripts/ci/run-fast-checks.sh (passed)

  • ./.build/debug/icloud-cli photos authorization --format json (exit 0, denied state, no access request)

No checks skipped.

Bootstrap Governance

  • Changes are scoped to the linked issue
  • Contributor or PR guidance changes are reflected in CONTRIBUTING.md, .github/PULL_REQUEST_TEMPLATE.md, and docs/bootstrap/onboarding.md when applicable
  • PR author enabled auto-merge where GitHub allows it, or GitHub plan-limit evidence/unavailable reason is recorded and the fallback merge-readiness policy applies
  • No real secrets, runtime auth, or machine-local env files are committed

No contributor guidance changed. Auto-merge will be enabled after stacked dependencies merge.

Flow Contract

  • Owner lane: Hermes native provider
  • Repair owner: jmcte
  • Autonomy class: bounded implementation
  • Risk class: high-sensitivity metadata; read-only

Flow Merge Readiness

  • Every blocker has a next actor and next action
  • No active blocking requested changes remain
  • Non-author approval is present when required
  • PR author enabled auto-merge where GitHub allows it, or recorded why it is unavailable/unsafe

Next actor: independent reviewer after #101 and #102 merge.

Merge Automation

  • PR author enabled auto-merge with gh pr merge --auto --squash, or the reason it is unavailable/unsafe is noted below

This draft is stacked on the #89 provider manifest and #90 SQLite snapshot foundation. It can be rebased and enabled for auto-merge after those land.

Notes

  • PhotoKit does not expose reliable local/cloud availability without attempting a media request, so that fact is explicitly unknown.
  • Derived observations are separate from source facts and remain empty; no OCR or classification occurs.

@jmcte
jmcte marked this pull request as ready for review July 13, 2026 07:29
@jmcte
jmcte requested a review from a team as a code owner July 13, 2026 07:29
@jmcte
jmcte enabled auto-merge (squash) July 13, 2026 07:30

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exact head passes swift test (117 tests), but this PR must be rebased onto current main before approval. Live main is d7f88dc01bd283b53c55fa59ed63f43f212a50a5; GitHub reports this PR unmergeable/dirty, and a local merge-tree reproduces conflicts in CommandLine.swift, CommandRunner.swift, ProviderManifest.swift, and provider-manifest tests. Resolve the rebase while preserving main's providers external-manifest support and combining this PR's PhotoKit/provider capability changes, then rerun CI on the rebased head.

@Hermes-omt Hermes-omt added the state:needs-repair PR needs repair before review can proceed. label Jul 17, 2026
@daedalus-omt
daedalus-omt force-pushed the codex/issue-94-photokit-photos branch from 48beb71 to 21274dd Compare July 19, 2026 01:45
@athena-omt athena-omt added status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. state:waiting-checks Waiting for CI/check status to settle. and removed state:needs-repair PR needs repair before review can proceed. labels Jul 19, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes: photos list --photos-library PATH parses and retains PATH, but CommandRunner only uses it inside if options.degradedFilesystem; without that flag it calls PhotoKit against the system library. This silently reads a different library than the caller explicitly selected. Reject --photos-library unless --degraded-filesystem is present (or make the option select the intended source), and add a CommandRunner-level regression test.

@Hermes-omt Hermes-omt added state:needs-repair PR needs repair before review can proceed. and removed status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. state:waiting-checks Waiting for CI/check status to settle. labels Jul 19, 2026
@jmcte

jmcte commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Implemented the outstanding review repair in 70f13f6.

  • photos list --photos-library PATH now fails unless --degraded-filesystem is also present, so a caller-selected source cannot be silently ignored by the PhotoKit path.
  • Added a CommandRunner regression for the invalid option combination.
  • The earlier rebase request is already satisfied: this branch is currently mergeable against main.

Verified locally:

  • swift test — 121 tests passed
  • bash scripts/ci/run-fast-checks.sh — passed (policy, security/fixture checks, coverage/mutation checks, debug/release builds)
  • git diff --check

Please re-review when convenient.

@athena-omt athena-omt added status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. and removed state:needs-repair PR needs repair before review can proceed. labels Jul 21, 2026
Hermes-omt
Hermes-omt previously approved these changes Jul 22, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 70f13f6. The prior Photo Library override repair is present, main is current and mergeable, and CI is green. Hermes validation passed: swift test, scripts/ci/run-fast-checks.sh, and git diff --check. Approve.

@Hermes-omt Hermes-omt removed status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. labels Jul 22, 2026

@athena-omt athena-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$P1: The new withSnapshot routine does not create a coherent SQLite snapshot of a live WAL-mode database. It copies the main database at lines 108-110 and then separately tests/copies -wal and -shm at 111-115, with no read transaction, SQLite backup API, or lock held on the source. A writer may checkpoint/truncate or append between those operations, so the copied main/WAL pair can represent different database states; SQLite can then return stale/inconsistent results or fail to open the purported snapshot. This affects the newly migrated Messages and Safari history reads. Please create the copy through SQLite while holding a read snapshot (for example using the backup API / VACUUM INTO in a read transaction), rather than copying live files independently, and add a race/regression test covering checkpoint or WAL mutation during snapshot creation.

@athena-omt athena-omt added the state:needs-repair PR needs repair before review can proceed. label Jul 22, 2026
@jmcte

jmcte commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the current-head review request.

The snapshot engine now uses SQLite VACUUM INTO to materialize one coherent snapshot rather than independently copying a database and WAL. Added coverage for a live WAL-backed source and documented the invariant.

Local verification:

  • swift test --filter snapshotQuery
  • swift test (122 tests)

@jmcte
jmcte force-pushed the codex/issue-94-photokit-photos branch from 8d58485 to b4c2dd4 Compare July 23, 2026 00:00
@athena-omt athena-omt added the review:athena Athena review governance requested. label Jul 23, 2026
Hermes-omt
Hermes-omt previously approved these changes Jul 23, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head b4c2dd4 on Hermes. The live-WAL coherence repair now uses SQLite VACUUM INTO, with a single-file live-WAL regression test. Branch is current, clean, and mergeable against main; no inline review threads are present. Local swift test and git diff --check passed; CI Gate and PR Checks succeeded. Approve.

@Hermes-omt Hermes-omt removed status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. labels Jul 23, 2026
@hephaestus-omt
hephaestus-omt force-pushed the codex/issue-94-photokit-photos branch from b4c2dd4 to d199b09 Compare July 25, 2026 01:20
@athena-omt athena-omt added status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. state:waiting-checks Waiting for CI/check status to settle. and removed state:waiting-checks Waiting for CI/check status to settle. labels Jul 25, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head d199b09 on Hermes. The prior PhotoKit option and SQLite live-WAL concerns are repaired: caller-selected Photos libraries require the explicit degraded path, and snapshots use SQLite VACUUM INTO with live-WAL coverage. Local swift test and git diff --check passed. CI Gate and PR Checks are successful; the PR is clean and auto-merge is enabled. Approve.

@Hermes-omt Hermes-omt removed status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. labels Jul 25, 2026
@hephaestus-omt
hephaestus-omt force-pushed the codex/issue-94-photokit-photos branch from d199b09 to 3a4bb22 Compare July 25, 2026 06:34
@athena-omt athena-omt added the state:waiting-checks Waiting for CI/check status to settle. label Jul 25, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 3a4bb221b72b22ef2e4a997e70009716e6673f59 on Hermes. The PhotoKit-first path, explicit filesystem fallback, and coherent SQLite VACUUM INTO snapshot path are correctly wired and covered by adjacent tests. swift test and git diff --check pass; live CI Gate and PR Checks are green. No actionable issues found.

@Hermes-omt Hermes-omt removed the state:waiting-checks Waiting for CI/check status to settle. label Jul 25, 2026
@daedalus-omt
daedalus-omt force-pushed the codex/issue-94-photokit-photos branch from 3a4bb22 to 1f6e93c Compare July 25, 2026 15:58
@athena-omt athena-omt added the state:waiting-checks Waiting for CI/check status to settle. label Jul 25, 2026

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 1f6e93c on Hermes. PhotoKit-first reads, explicit filesystem fallback, and coherent SQLite VACUUM INTO snapshots are correctly wired and covered. swift test and git diff --check passed; CI Gate and PR Checks are successful. No actionable issues found.

@Hermes-omt Hermes-omt removed the state:waiting-checks Waiting for CI/check status to settle. label Jul 25, 2026

@athena-omt athena-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head re-review at 1f6e93cb9e2e4b15e6146550615c8bcf618665c6APPROVE.

photos list is now PhotoKit-first and fail-closed. Verified on exact head:

  • PhotoKitProvider never requests authorization or media: authorization() reports requestsAccess=false; assets() throws unless state is .authorized/.limited — there is no silent filesystem fallback.
  • Metadata-only: fetchAssets reads PHAsset facts (id, filename, media type, dates, pixel dimensions, favorite/hidden, album names); availability is honestly reported as .unknown; no pixels/thumbnails are requested.
  • Filesystem path is opt-in via --degraded-filesystem, and --photos-library requires it (parser-enforced and error-tested).
  • Provenance: degraded rows tagged photos-library-filesystem / degraded=true; PhotoKit rows tagged photokit / degraded=false.
  • PermissionsDoctor now probes PhotoKit authorization (photokit-<state>) instead of a filesystem path probe.
  • The SQLite snapshot engine on this head also uses SQLite-coordinated VACUUM INTO (no independent file copy).
  • Regression tests cover: never-prompt, no-media, evidence/provenance, explicit-flag requirement, --photos-library rejection without the flag, and the doctor probe.
  • Required checks green on exact head (CI Gate, PR Checks). git diff --check clean.

The earlier CHANGES_REQUESTED review was against stale head 70f13f6a; superseded by this approval.

Hermes-omt and others added 3 commits July 28, 2026 11:29
Return evidence-backed logical asset facts through bounded PhotoKit reads without media requests or downloads. Keep the filesystem walk as an explicit degraded fallback and expose non-prompting authorization diagnostics.
Require explicit degraded filesystem mode when callers supply a Photo Library path, with CommandRunner coverage for the invalid combination.

Signed-off-by: John McChesney TenEyck Jr <59268465+jmcte@users.noreply.github.com>
@pheidon
pheidon force-pushed the codex/issue-94-photokit-photos branch from 1f6e93c to 990bbb1 Compare July 28, 2026 10:29
@jmcte
jmcte merged commit 3469c2d into main Jul 28, 2026
5 checks passed
@jmcte
jmcte deleted the codex/issue-94-photokit-photos branch July 28, 2026 10:32
jmcte pushed a commit that referenced this pull request Jul 28, 2026
Merge origin/main (post-#110/#107) into the SQLite-snapshot branch.
Conflicts in the snapshot surface resolved in favor of the PR's repairs:
- SQLiteSnapshotQuery.swift: keep resolvedSource (symlink resolution) feeding
  VACUUM INTO so live/symlinked WAL stores snapshot coherently.
- SQLiteSnapshotQueryTests.swift: keep both the PR symlink-coherent-WAL test and
  main's single-file-WAL test, plus the production timeout-floor test.
- docs/sqlite-snapshots.md: keep production timeout minimums (500ms busy / 5s process).

swift build clean; swift test --filter SQLiteSnapshot 10/10 passed on macOS (Hermes).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v2: Make Photos a PhotoKit-first evidence-backed provider

3 participants