feat(auth): application-layer auth foundation - #197
Open
KinshukSS2 wants to merge 9 commits into
Open
Conversation
KinshukSS2
force-pushed
the
feat/auth-foundation-phase0
branch
from
July 16, 2026 18:03
c316cb9 to
e1e7bf4
Compare
Closes Issue istSOS#28 — eliminates two-step user provisioning. After POST /Users, a newly created user had no RLS policy and could not access any data until an administrator separately called POST /Policies. This commit fixes that by automatically calling the appropriate policy function inside the same transaction as user creation. Changes: - api/app/v1/endpoints/create/user.py * Add module-level _POLICY_FN_MAP (viewer/editor/obs_manager/sensor). * Capture app_role before get_db_role_for_rbac() remaps it, so the correct policy function can be dispatched. * After GRANT, call sensorthings.<role>_policy([username], policyname) with policyname = '{username}_default'. Administrator is skipped — admins bypass RLS by privilege, not by policy. * Policy functions already exist in the DB (istsos_auth.sql); no migration required. - api/app/v1/endpoints/functions.py * Add docstrings to _validate_role_identifier() and set_role(). - api/app/v1/endpoints/create/data_array_observation.py * Import shared set_role() helper (was already using the correct upstream version; this import makes the dependency explicit). - api/tests/test_rls_policy_creation.py (new) * Tests: correct policy function per role, administrator exclusion, naming convention, users_ as text[]. - api/tests/test_rbac_set_role_safety.py (new) * Tests: identifier validation, injection rejection, shared helper usage in data_array_observation.
- Add auth_provider and external_sub_id columns to sensorthings."User"
via idempotent migration (001_identity_linking.sql) with a partial
unique index on (auth_provider, external_sub_id) WHERE auth_provider
IS NOT NULL, so local password users are completely unaffected.
- Introduce PENDING_ROLE sentinel in rbac_roles.py. The 'pending' state
is intentionally absent from VALID_RBAC_ROLES so it can never be
assigned through the public API; existing role validation is unchanged.
- Gate pending accounts in get_current_user() (oauth.py): after the DB
lookup, any user with role='pending' immediately receives HTTP 403
'Account pending admin activation' before any SET ROLE or handler
body is reached.
- Add oidc_user_crud.py with create_pending_oidc_user() and
get_user_by_provider_sub(). The insert function hardcodes role to
PENDING_ROLE and contains zero DDL (no CREATE ROLE / CREATE USER),
giving new OIDC accounts zero PostgreSQL footprint until activation.
- Add POST /Users/{id}/activate endpoint (activate_user.py), restricted
to administrators. Runs UPDATE role, CREATE ROLE NOLOGIN IN ROLE,
GRANT, and RLS policy assignment inside a single transaction so a
failed step leaves the user still 'pending' with no partial state.
- Register activate_user router in api.py inside the AUTHORIZATION guard.
Local password users (POST /Users) are completely unaffected; no changes
were made to create/user.py.
Relates-to: GSoC 2026 Identity Linking architecture
- Add PasswordUpdateRequest Pydantic v2 schema (models/password.py)
enforcing: min 12 chars, at least 1 uppercase, at least 1 digit.
Violations surface as HTTP 422 before any DB is touched.
- Add update_local_password() CRUD function (db/password_crud.py):
1. Fetch user row by ID → 404 if missing.
2. OIDC guard: block auth_provider IS NOT NULL users with HTTP 400
'External identities cannot update passwords locally'.
3. Verify current_password via asyncpg.connect() (PostgreSQL auth layer)
→ 401 on InvalidPasswordError. No Python-side passlib used.
4. Execute ALTER USER <username> WITH ENCRYPTED PASSWORD <new_password>
using pg_quote_ident / pg_quote_literal to prevent injection.
- Add PATCH /Users/{id}/password endpoint (update/password.py):
owner-or-admin guard; returns 204 No Content on success.
- Register update_password router in api.py inside AUTHORIZATION guard.
- Add test_password_update.py (9 tests, all pass, no live DB required):
schema: valid, too-short, no-uppercase, no-digit
crud: 404, 400 OIDC block, 401 wrong password, 204 ALTER USER issued
endpoint: 403 non-owner/non-admin guard
Depends on: feat/identity-linking-jit-provisioning (requires auth_provider column)
…le-JWT fix
- Add RoleUpdateRequest Pydantic v2 schema (models/role.py):
delegates to validate_rbac_role(); blocks 'administrator' (bootstrap-only)
and 'pending' (internal state) with HTTP 422 before any DB is touched.
Docstring explains the security boundary explicitly.
- Add update_user_role() CRUD function (db/role_crud.py):
All mutations run inside a single asyncpg transaction (FOR UPDATE lock):
1. 404 if user not found.
2. 400 if user is in 'pending' waiting room.
3. No-op early return if current_role == new_role (no DDL issued).
4. 409 if demoting the last administrator (lockout guard).
5. UPDATE sensorthings."User" SET role = new_role.
6. REVOKE <old_pg_group_role> / GRANT <new_pg_group_role> only when the
underlying PostgreSQL group role changes (e.g. viewer→obs_manager).
viewer→editor shares the same 'user' PG role — no DDL issued.
pg_quote_ident used for all identifier interpolation.
- Add PATCH /Users/{id}/role endpoint (update/role.py):
administrator-only guard at router layer; returns 204 No Content.
- Register update_role router in api.py inside AUTHORIZATION guard.
- Add comment to get_current_user() in oauth.py documenting that role is
fetched live from the DB on every request (not from the JWT payload),
eliminating stale-JWT vulnerabilities after role changes. No logic change.
- Add test_role_reassignment.py (12 tests, all pass, no live DB needed):
schema: valid, administrator/pending/unknown blocked
crud: 404, 400 pending, no-op, 409 last-admin, REVOKE+GRANT, no-DDL
endpoint: 403 non-admin guard
Depends on: feat/password-updates (stacked)
…dentials
Users are strictly application-level entities managed via sensorthings."User".
The backend connects to PostgreSQL through a single master service account;
individual users have no PostgreSQL login roles.
- Add shared POLICY_FN_MAP in rbac_roles.py as single source of truth for
RLS policy dispatch (used by create/user.py and activate_user.py)
- Role reassignment (PATCH /Users/{id}/role) is a pure UPDATE on User.role;
last-admin lockout locks all admin rows via SELECT … FOR UPDATE before
counting to prevent concurrent demotion race condition
- User activation (POST /Users/{id}/activate) updates User.role and applies
the corresponding RLS policy function — no PostgreSQL DDL
- User creation stores bcrypt hash in User.password via parameterised UPDATE
Removed: CREATE USER, CREATE ROLE, REVOKE, GRANT, ALTER USER DDL.
Refs: istSOS#190
BREAKING: set_role() now maps app-layer roles to PG group roles via SET LOCAL ROLE instead of SET ROLE <username>. All 72 RESET ROLE instances deleted across 41 endpoint files. Refactor 1 — functions.py::set_role(): - Maps viewer/editor → 'user', sensor/obs_manager → 'sensor', etc. - SET LOCAL ROLE (transaction-scoped, auto-reverts on COMMIT/ROLLBACK) - Removed inner connection.transaction() nested savepoint - Eliminated entire RESET ROLE call class (pool leak prevention) Refactor 2 — password_crud.py: - Removed asyncpg.connect() credential verification - Removed ALTER USER … WITH ENCRYPTED PASSWORD DDL - Modern path: passlib/bcrypt verify + UPDATE User.password - Legacy fallback: get_auth_connection() for NULL password JIT migration Refactor 3 — role_crud.py: - Removed REVOKE/GRANT DDL block (lines 167-196) - Role reassignment is pure UPDATE sensorthings."User" SET role - Removed _ADMIN_PG_ROLE, DB_ROLE_BY_RBAC_ROLE, pg_quote_ident imports Refactor 4 — Test suite alignment: - test_set_role_sql_safety: asserts SET LOCAL ROLE + group role mapping - test_rbac_set_role_safety: parametrised 5 app roles → PG group roles - test_password_update: asserts UPDATE + bcrypt hash (not ALTER USER) - test_role_reassignment: asserts UPDATE only (not REVOKE/GRANT) - test_policy_role_switch: asserts SET LOCAL ROLE + zero RESET ROLE 49 files changed, 298 insertions(+), 442 deletions(-) 63/63 tests passing.
update/user.py lines 119-135 contained live REVOKE/GRANT statements that were missed in the PR5 cleanup pass. These operated on individual usernames as PostgreSQL role identifiers, which no longer exist under the app-layer credential model. Remove the entire DDL block. Role changes are reflected in the sensorthings."User".role UPDATE above; set_role() maps the new role to its PG group role dynamically at request time.
## Summary
Completes the authentication foundation required before building any
new access-control features. Two self-contained changes:
1. database/migrations/002_add_password_status.sql
2. api/app/oauth.py — rewrite authenticate_user()
---
## 1. Migration: 002_add_password_status.sql
Adds two columns to sensorthings."User" inside the existing
custom.authorization guard block (mirrors migration 001):
• password VARCHAR(255) DEFAULT NULL
Stores the passlib/bcrypt hash of the user's local credential.
NULL signals a legacy (pre-migration) account; on next login the
pg_authid JIT fallback fires and backfills this column so
subsequent logins never touch pg_authid again.
• status VARCHAR(50) DEFAULT 'active'
Account lifecycle flag. 'active' is the default so all existing
rows are completely unaffected. Future values: 'suspended',
'deleted'. Application-layer enforcement is a follow-up task.
Both ALTER TABLE statements use ADD COLUMN IF NOT EXISTS so the
migration is idempotent and safe to re-run against instances that
already received the columns via a prior manual hotfix.
---
## 2. authenticate_user() — bcrypt-first with JIT pg_authid fallback
Replaces the legacy pg_authid-only authentication flow with a
three-step process:
Step 1 — Fetch from sensorthings."User"
SELECT id, username, role, password WHERE username = $1.
Unknown users return None immediately; we never attempt pg_authid
for users not present in the application-layer table.
Step 2 — Bcrypt verify (modern path, password IS NOT NULL)
pwd_context.verify() is dispatched via asyncio.to_thread() to
avoid blocking the event loop with bcrypt's intentional CPU cost.
Correct hash → return user dict. Wrong hash → return None.
Step 3 — pg_authid JIT fallback (legacy path, password IS NULL)
get_auth_connection() attempts a raw asyncpg.connect() to let
PostgreSQL validate via pg_authid.
• Failure → return None.
• Success → asyncio.to_thread(pwd_context.hash, password) computes
the bcrypt hash then writes it via the write pool
(POSTGRES_PORT_WRITE pattern from role_crud / password_crud).
Backfill failure is caught, logged, and swallowed — login still
succeeds (best-effort JIT migration, never blocks the user).
Circular import resolution
pwd_context lives in password_crud.py which already imports
get_auth_connection from oauth.py. Both symbols are imported
lazily inside the function body to break the cycle, following the
identical pattern already used in password_crud.py line 115.
---
Breaking changes: none.
Existing users with password IS NULL continue to log in as before;
they are transparently migrated on first login.
Users with a bcrypt hash no longer require a pg_authid LOGIN role.
…ation errors passlib 1.7.4 is incompatible with bcrypt >= 4.1 due to a wrap-bug detection test that passes a >72-byte secret, which bcrypt 4+ rejects with ValueError. Pin bcrypt==4.0.1 — the last version that works with passlib 1.7.4 without triggering the 72-byte guard. A previous refactor left dangling 'if current_user is not None:' guards with no body before the return statement, causing Python to raise IndentationError at import time and crashing the entire API process. Removed the dead guard in each case — the 404 / not-found response should always be returned regardless of auth context. The outer exception handler already enforces auth context where needed. Files fixed: api/app/v1/endpoints/create/bulk_observation.py (ValueError catch) api/app/v1/endpoints/delete/observation.py (404 guard) api/app/v1/endpoints/update/historical_location.py (404 guard) api/app/v1/endpoints/update/location.py (404 guard) api/app/v1/endpoints/update/observation.py (404 guard) api/app/v1/endpoints/update/observed_property.py (404 guard) api/app/v1/endpoints/update/thing.py (404 guard)
KinshukSS2
force-pushed
the
feat/auth-foundation-phase0
branch
from
July 21, 2026 13:33
e1e7bf4 to
c123d62
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Branch:
feat/auth-foundation-phase0→fix/post-pivot-cleanupOverview
This PR is Phase 0 of the auth-foundation plan agreed in the STAC/ODRL architecture review. It is a prerequisite for every access-control feature that follows.
It delivers two self-contained changes:
database/migrations/002_add_password_status.sqlpasswordandstatuscolumns tosensorthings."User"api/app/oauth.py→authenticate_user()Change 1 — Migration
002_add_password_status.sqlAdds two columns to
sensorthings."User"wrapped in the project-standardcurrent_setting('custom.authorization')guard block, withSET ROLE "administrator"before andRESET ROLEafter (identical pattern to001_identity_linking.sql).passwordsemantics:NULL→ legacy (pre-migration) account. The JIT fallback inauthenticate_user()fires on next login and backfills this column.NOT NULL→ modern account. The bcrypt path inauthenticate_user()is used exclusively; no pg_authid connection is ever opened.statussemantics:'active'(default) → normal login permitted. All existing rows receive this value automatically — zero disruption.'suspended','deleted') will be enforced at the application layer; no DB constraint is added here to keep the migration forward-compatible.Both statements use
ADD COLUMN IF NOT EXISTS— the migration is fully idempotent and safe to re-run against instances that received the columns via a prior manual hotfix.Change 2 —
authenticate_user()rewriteBefore (legacy):
One path. Requires every user to have a PostgreSQL
LOGINrole. Incompatible with the application-layer pivot.After (this PR):
Key implementation decisions:
asyncio.to_threadfor bcryptpwd_contextpassword_crud.pyalready importsget_auth_connectionfromoauth.py; a module-level import ofpwd_contextinoauth.pywould create a circular dependency. The lazy import pattern is already used inpassword_crud.pyline 115POSTGRES_PORT_WRITEpattern for backfillrole_crud.pyandpassword_crud.py; honours read-replica topologyTesting
This PR does not add new test files (test harness for the login flow with bcrypt accounts is tracked separately). The existing
api/tests/test_password_update.pycoversupdate_local_password()which exercises the samepwd_contextand write-pool path.Manual verification steps:
status = 'active'andpassword = NULL.pg_stat_activity).Nonereturned, 401 issued.Nonereturned, 401 issued (no pg_authid probe).Migration from upstream
origin/mainusers applying this migration to an existing database will see no disruption:"User"rows gainpassword = NULLandstatus = 'active'.LOGINrole is created or dropped.Files Changed
Checklist
ADD COLUMN IF NOT EXISTS)001_identity_linking.sqlstructural pattern exactlyasyncio.to_threadused for all blocking bcrypt workLOGINroles created or requiredfeat/auth-foundation-phase0based onfix/post-pivot-cleanup(dc2d07b)