Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
3344016
docs: record playlist ingest design approval (TASK-12109)
rmusser01 Jul 12, 2026
e9ca062
docs: plan per-video playlist ingestion (TASK-12110 TASK-12111)
rmusser01 Jul 12, 2026
d7b558f
chore: track playlist ingestion implementation (TASK-12112 TASK-12113)
rmusser01 Jul 12, 2026
04fb636
docs: start playlist ingestion backend implementation (TASK-12112)
rmusser01 Jul 12, 2026
30f391b
feat: add playlist ingest persistence schema (TASK-12112)
rmusser01 Jul 12, 2026
d1b56a9
fix: align playlist ingest SQLite primary keys (TASK-12112)
rmusser01 Jul 12, 2026
cda26cb
fix: harden playlist ingest persistence contracts (TASK-12112)
rmusser01 Jul 12, 2026
4f5a715
fix: reject coerced playlist ingest metadata inputs (TASK-12112)
rmusser01 Jul 12, 2026
0487f7c
docs: record playlist ingest schema completion (TASK-12112)
rmusser01 Jul 12, 2026
744def1
feat: persist playlist ingest resources (TASK-12112)
rmusser01 Jul 12, 2026
b996219
fix: enforce playlist resource snapshot boundaries (TASK-12112)
rmusser01 Jul 12, 2026
bc6ca9c
fix: harden playlist store concurrency and limits (TASK-12112)
rmusser01 Jul 12, 2026
ef36ba3
docs: complete playlist ingest storage stage (TASK-12112)
rmusser01 Jul 12, 2026
0977881
feat: run playlist inspection as bounded jobs (TASK-12112)
rmusser01 Jul 12, 2026
767c81b
fix: preserve unavailable playlist entries (TASK-12112)
rmusser01 Jul 12, 2026
0c21943
fix: detect yt-dlp unavailable placeholders (TASK-12112)
rmusser01 Jul 12, 2026
b378253
fix: harden playlist preflight process finalization (TASK-12112)
rmusser01 Jul 12, 2026
e696503
fix: fence playlist preflight snapshots by lease (TASK-12112)
rmusser01 Jul 12, 2026
bb65edb
fix: guard playlist preflight failure snapshots (TASK-12112)
rmusser01 Jul 12, 2026
0d3992a
fix: block active playlist snapshot failures (TASK-12112)
rmusser01 Jul 13, 2026
e19cdcd
docs: record bounded playlist preflight completion (TASK-12112)
rmusser01 Jul 13, 2026
65ab771
feat: expose playlist preflight resources (TASK-12112)
rmusser01 Jul 13, 2026
dcb1132
fix: enforce playlist preflight read boundaries (TASK-12112)
rmusser01 Jul 13, 2026
7178eab
docs: record playlist preflight API verification (TASK-12112)
rmusser01 Jul 13, 2026
1a9ccce
fix: harden playlist preflight admission and publication (TASK-12112)
rmusser01 Jul 13, 2026
ed23a45
fix: normalize playlist cleanup expiry checks (TASK-12112)
rmusser01 Jul 13, 2026
3b7bb4f
docs: complete playlist preflight resource stage (TASK-12112)
rmusser01 Jul 13, 2026
b91aa25
feat: validate playlist ingest runs (TASK-12112)
rmusser01 Jul 13, 2026
36a6f5c
fix: defer playlist run override validation (TASK-12112)
rmusser01 Jul 13, 2026
517c09c
fix: harden playlist run authority boundaries (TASK-12112)
rmusser01 Jul 13, 2026
dd1241e
fix: correct PostgreSQL playlist resolver SQL (TASK-12112)
rmusser01 Jul 13, 2026
3f1e842
feat: resolve playlist duplicate actions (TASK-12112)
rmusser01 Jul 13, 2026
d7fb395
fix: complete playlist collection planning (TASK-12112)
rmusser01 Jul 13, 2026
fd2d102
fix: make playlist collection actions fail closed (TASK-12112)
rmusser01 Jul 13, 2026
8d49790
fix: harden playlist duplicate action recovery (TASK-12112)
rmusser01 Jul 13, 2026
fa0a736
fix: reconcile pending playlist duplicate actions (TASK-12112)
rmusser01 Jul 13, 2026
09b4bde
fix: finalize playlist recovery boundaries (TASK-12112)
rmusser01 Jul 13, 2026
2bb61c9
fix: sanitize playlist reconciliation reads (TASK-12112)
rmusser01 Jul 13, 2026
21c48c8
feat: bind media jobs to ingest occurrences (TASK-12112)
rmusser01 Jul 13, 2026
389212c
fix: harden playlist job submission recovery (TASK-12112)
rmusser01 Jul 13, 2026
9e3073f
fix: recover pending playlist submissions (TASK-12112)
rmusser01 Jul 13, 2026
695f08f
fix: preserve playlist submission reservations (TASK-12112)
rmusser01 Jul 13, 2026
1b11771
fix: propagate playlist retry backpressure (TASK-12112)
rmusser01 Jul 13, 2026
adf2143
fix: harden playlist job binding lifecycle (TASK-12112)
rmusser01 Jul 13, 2026
3bdfe78
fix: sanitize oversized playlist binding integers (TASK-12112)
rmusser01 Jul 13, 2026
8861bbf
fix: finalize playlist submission lifecycle safety (TASK-12112)
rmusser01 Jul 13, 2026
479ecd5
feat: track playlist ingest runs (TASK-12112)
rmusser01 Jul 13, 2026
794fea6
feat: finish playlist ingest rollout safety (TASK-12112)
rmusser01 Jul 13, 2026
b86d716
feat: add playlist ingest v2 client (TASK-12113)
rmusser01 Jul 13, 2026
e6b2acb
fix: require playlist inspection before queueing (TASK-12113)
rmusser01 Jul 13, 2026
ad1b392
feat: show complete virtualized playlist previews (TASK-12113)
rmusser01 Jul 14, 2026
dd3ac85
feat: preserve playlist occurrence identity in review (TASK-12113)
rmusser01 Jul 14, 2026
82feb39
feat: submit quick ingest through shared runs (TASK-12113)
rmusser01 Jul 14, 2026
4bf0ad7
docs: record Task 5 completion (TASK-12113)
rmusser01 Jul 14, 2026
bdf1f6d
feat: align extension playlist run transport (TASK-12113)
rmusser01 Jul 14, 2026
5fc804c
docs: record Task 6 completion (TASK-12113)
rmusser01 Jul 14, 2026
e84ba7e
feat: show per-occurrence ingest lifecycle (TASK-12113)
rmusser01 Jul 14, 2026
3dda483
docs: record playlist lifecycle implementation (TASK-12113)
rmusser01 Jul 14, 2026
8a9fc44
feat: persist quick ingest runs in indexeddb (TASK-12113)
rmusser01 Jul 15, 2026
a65419b
docs: close Task 8 quick ingest persistence (TASK-12113)
rmusser01 Jul 15, 2026
064a3ad
test: verify shared playlist ingest experience (TASK-12113)
rmusser01 Jul 15, 2026
d8d3bff
docs: close shared playlist ingest task (TASK-12113)
rmusser01 Jul 15, 2026
a2b1809
fix: reconcile playlist ingest with current dev contracts (TASK-12113)
rmusser01 Jul 15, 2026
981a2d1
docs: link playlist ingest draft PR (TASK-12113)
rmusser01 Jul 15, 2026
003a673
fix: make playlist run creation idempotent (TASK-12970)
rmusser01 Jul 15, 2026
fca2543
fix: discard stale playlist collection plan (TASK-12970)
rmusser01 Jul 15, 2026
8799164
fix: recover committed playlist collection plan (TASK-12970)
rmusser01 Jul 15, 2026
1b7fdbe
fix: fence playlist collection cleanup ownership (TASK-12970)
rmusser01 Jul 15, 2026
8007e92
fix: close playlist collection fencing gaps (TASK-12970)
rmusser01 Jul 15, 2026
2ec9d07
fix: validate recovered playlist collection identity (TASK-12970)
rmusser01 Jul 15, 2026
23581bb
fix: replay playlist run creation by session (TASK-12970)
rmusser01 Jul 15, 2026
c7c31ce
fix: harden playlist session identity (TASK-12970)
rmusser01 Jul 15, 2026
5fd526f
test: type playlist fallback message mock (TASK-12970)
rmusser01 Jul 15, 2026
4fcfd9f
fix: enforce playlist ingest postgres rls (TASK-12970)
rmusser01 Jul 15, 2026
5391051
fix: harden playlist ingest postgres rls (TASK-12970)
rmusser01 Jul 15, 2026
91fb12d
fix: close playlist postgres rls review gaps (TASK-12970)
rmusser01 Jul 15, 2026
a449675
fix: close remaining postgres rls review gaps (TASK-12970)
rmusser01 Jul 15, 2026
95bbd61
fix: enforce dedicated postgres rls privileges (TASK-12970)
rmusser01 Jul 15, 2026
d35f125
fix: enforce database-wide postgres rls isolation (TASK-12970)
rmusser01 Jul 15, 2026
51b9eac
fix: retry playlist job cancellation (TASK-12970)
rmusser01 Jul 15, 2026
416993b
fix: route playlist jobs from run options (TASK-12970)
rmusser01 Jul 15, 2026
828fdc4
fix: fence playlist cancellation bindings (TASK-12970)
rmusser01 Jul 15, 2026
ef64e16
fix: require exact cancellation binding values (TASK-12970)
rmusser01 Jul 15, 2026
4b02cde
docs: finalize playlist review remediation (TASK-12970)
rmusser01 Jul 15, 2026
f303f5e
fix: address playlist ingest PR review (TASK-12971)
rmusser01 Jul 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions Docs/Code_Documentation/Jobs_Module.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,11 @@ Audio Studio job payloads are sanitized before persistence. Client requests must
- `JOBS_INTEGRITY_SWEEP_ENABLED` (default false), `JOBS_INTEGRITY_SWEEP_INTERVAL_SEC` (default 60), `JOBS_INTEGRITY_SWEEP_FIX` (default false).
- Periodically flags (and optionally fixes) impossible states like leases on non-processing jobs and expired processing leases.
- Postgres RLS (optional):
- `JOBS_PG_RLS_ENABLE` (default false): Enable row-level security policies that scope access to domains in `current_setting('app.domain_allowlist')`.
- To scope a connection/session, set the allowlist before issuing queries/updates (example):
- `SELECT set_config('app.domain_allowlist', 'chatbooks,prompt_studio', true);`
- The policies will then allow access only to rows where `domain` is in that list.
- `JOBS_PG_RLS_ENABLE` (default false): Enable row-level security policies that scope Jobs access by domain/owner and playlist authority tables by owner.
- `JOBS_PG_RLS_ROLE` must be a dedicated `NOLOGIN`, non-superuser group role without `BYPASSRLS` or parent-role memberships. Across every non-system schema, the installer revokes all direct schema, table, and sequence privileges from that role, then grants only current-schema `USAGE` plus the enumerated Jobs/playlist objects. Migration removes any custom direct grants on this dedicated role; place unrelated privileges on another role. Installation fails if `PUBLIC`, inherited, or ownership-derived privileges expose an unrelated object, grant `CREATE` on a non-system schema, or exceed the allowlist ceiling (`SELECT`/`INSERT`/`UPDATE`/`DELETE` on tables and `USAGE`/`SELECT` on sequences); it does not mutate `PUBLIC`. The application login receives membership and `JobManager` assumes the role for managed connections.
- When both RLS settings are configured, managed connections fail closed if `SET ROLE` or any request-scoped RLS setting fails; queries never continue under the application login.
- Use `JobManager.rls_context(...)` around owner-scoped operations. It restores prior context across nesting and exceptions; playlist authority policies fail closed when `app.owner_user_id` is unset or blank.
- RLS is a trusted-application query-bug guardrail. It is not a defense against direct database access, SQL injection, a compromised application login, or database owner/superuser access; enforce network and credential least privilege separately.
- Metrics/Tracing buckets:
- `JOBS_DURATION_BUCKETS`: CSV of float seconds for `duration_seconds` histogram buckets.
- `JOBS_QUEUE_LATENCY_BUCKETS`: CSV of float seconds for `queue_latency_seconds` histogram buckets.
Expand Down Expand Up @@ -441,6 +442,7 @@ jm.fail_job(job["id"], error="boom", retryable=True, worker_id=worker_id, lease_
- Emits text/event-stream with incremental IDs; clients can resume by passing `after_id`.
- Requires `JOBS_EVENTS_OUTBOX=true` to persist events; otherwise events are process-local only.
- Admin endpoints set per-request Postgres RLS context automatically when enabled.
- Playlist ingestion store and worker paths set owner-scoped RLS context before opening managed Postgres connections; only genuinely global operations use admin context.
- Response: `{ non_processing_with_lease: int, processing_expired: int, fixed: int }`
- When `fix=true`, clears stale lease fields on non-processing rows, and re-queues expired processing rows.

Expand Down
9 changes: 5 additions & 4 deletions Docs/Published/Code_Documentation/Jobs_Module.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,11 @@ Connector sync jobs are not only deduped at Jobs level. They also reserve a sour
- `JOBS_INTEGRITY_SWEEP_ENABLED` (default false), `JOBS_INTEGRITY_SWEEP_INTERVAL_SEC` (default 60), `JOBS_INTEGRITY_SWEEP_FIX` (default false).
- Periodically flags (and optionally fixes) impossible states like leases on non-processing jobs and expired processing leases.
- Postgres RLS (optional):
- `JOBS_PG_RLS_ENABLE` (default false): Enable row-level security policies that scope access to domains in `current_setting('app.domain_allowlist')`.
- To scope a connection/session, set the allowlist before issuing queries/updates (example):
- `SELECT set_config('app.domain_allowlist', 'chatbooks,prompt_studio', true);`
- The policies will then allow access only to rows where `domain` is in that list.
- `JOBS_PG_RLS_ENABLE` (default false): Enable row-level security policies that scope Jobs access by domain/owner and playlist authority tables by owner.
- `JOBS_PG_RLS_ROLE` must be a dedicated `NOLOGIN`, non-superuser group role without `BYPASSRLS` or parent-role memberships. Across every non-system schema, the installer revokes all direct schema, table, and sequence privileges from that role, then grants only current-schema `USAGE` plus the enumerated Jobs/playlist objects. Migration removes any custom direct grants on this dedicated role; place unrelated privileges on another role. Installation fails if `PUBLIC`, inherited, or ownership-derived privileges expose an unrelated object, grant `CREATE` on a non-system schema, or exceed the allowlist ceiling (`SELECT`/`INSERT`/`UPDATE`/`DELETE` on tables and `USAGE`/`SELECT` on sequences); it does not mutate `PUBLIC`. The application login receives membership and `JobManager` assumes the role for managed connections.
- When both RLS settings are configured, managed connections fail closed if `SET ROLE` or any request-scoped RLS setting fails; queries never continue under the application login.
- Use `JobManager.rls_context(...)` around owner-scoped operations. It restores prior context across nesting and exceptions; playlist authority policies fail closed when `app.owner_user_id` is unset or blank.
- RLS is a trusted-application query-bug guardrail. It is not a defense against direct database access, SQL injection, a compromised application login, or database owner/superuser access; enforce network and credential least privilege separately.
- Metrics/Tracing buckets:
- `JOBS_DURATION_BUCKETS`: CSV of float seconds for `duration_seconds` histogram buckets.
- `JOBS_QUEUE_LATENCY_BUCKETS`: CSV of float seconds for `queue_latency_seconds` histogram buckets.
Expand Down
Loading
Loading