Skip to content

refactor: Finalize App-Layer Auth Pivot & Deprecate Legacy DDL - #191

Open
KinshukSS2 wants to merge 7 commits into
istSOS:mainfrom
KinshukSS2:fix/post-pivot-cleanup
Open

refactor: Finalize App-Layer Auth Pivot & Deprecate Legacy DDL#191
KinshukSS2 wants to merge 7 commits into
istSOS:mainfrom
KinshukSS2:fix/post-pivot-cleanup

Conversation

@KinshukSS2

Copy link
Copy Markdown
Contributor

refactor: Finalize App-Layer Auth Pivot & Deprecate Legacy DDL

What Problem Does This Fix?

PRs 1–4 progressively pivoted from PostgreSQL LOGIN roles to Application-Layer Credentials (passlib/bcrypt in sensorthings."User"). Users no longer have individual PostgreSQL login roles - they are pure application-layer entities.

This branch finalizes the transition to Application-Layer Credentials. It safely deprecates the legacy PostgreSQL DDL commands from the prior architecture, ensuring that all database interactions are perfectly aligned with the new, stateless application-layer identity model.

This branch fixes all of it.


Changes

1. functions.py::set_role() — Rewritten

Before: SET ROLE "alice" — targets a non-existent per-user PG role.
After: SET LOCAL ROLE "user" — maps the application role to its PostgreSQL group role using DB_ROLE_BY_RBAC_ROLE from rbac_roles.py.

Role mapping:

App Role PG Group Role
viewer user
editor user
obs_manager sensor
sensor sensor
custom user
administrator administrator

SET LOCAL ROLE is transaction-scoped: it automatically reverts on COMMIT or ROLLBACK, including implicit rollback from async generator cancellation. This eliminates the entire class of pool-leak bugs from the old RESET ROLE pattern.

2. RESET ROLE — Deleted (72 instances, 41 files)

All 72 await connection.execute("RESET ROLE;") calls have been deleted across every endpoint file. With SET LOCAL ROLE, they are not just redundant — they were actively dangerous: in the streaming read path, RESET ROLE could be skipped if the client disconnected mid-stream, returning a connection with an elevated group role to the pool.

Files affected: all create/, delete/, update/, read/ endpoint files.

3. password_crud.py — DDL Removed

Before:

  1. asyncpg.connect(user=username, password=current_password) — connects as the user to verify
  2. ALTER USER {username} WITH ENCRYPTED PASSWORD '...' — DDL requiring CREATEROLE

After:

  1. pwd_context.verify(current_password, stored_hash) — bcrypt verify against User.password (using the existing passlib dependency established in PR 3; no new hashing libraries introduced)
  2. UPDATE sensorthings."User" SET password = $1 WHERE id = $2 — parameterised DML
  3. Legacy fallback: if User.password IS NULL (pre-migration), falls back to get_auth_connection() for pg_authid verification, then writes the bcrypt hash (completing JIT migration)

4. role_crud.py — REVOKE/GRANT Removed

Before: After updating User.role, the code issued REVOKE {old_pg_role} FROM {username} and GRANT {new_pg_role} TO {username}.
After: Role reassignment is a pure UPDATE sensorthings."User" SET role = $1. set_role() maps the role to its PG group role dynamically at request time — no DDL membership changes needed.

5. update/user.py — REVOKE/GRANT Removed (found during post-audit)

update/user.py lines 119–135 contained a live REVOKE/GRANT block that was missed by the initial cleanup pass (it was in the general PATCH /Users endpoint, not in db/role_crud.py). Fixed in the follow-up commit. Unused get_db_role_for_rbac and pg_quote_ident imports also removed.

6. create/policy.py, update/policy.py — Dead try/finally Removed

The RESET ROLE deletion left dangling if role_switched: pass blocks inside finally: clauses, causing IndentationError. Cleaned up by removing the entire role_switched / try/finally pattern.

7. Test Suite Aligned (5 files)

File Change
test_set_role_sql_safety.py Asserts SET LOCAL ROLE "user" for viewer, group mapping for all roles
test_rbac_set_role_safety.py Parametrised: 5 app roles → correct PG group roles
test_password_update.py test_success_executes_update_with_hash replaces test_success_executes_alter_user
test_role_reassignment.py test_different_pg_group_still_no_ddl replaces test_different_pg_role_triggers_revoke_grant
test_policy_role_switch.py Asserts SET LOCAL ROLE "administrator", asserts zero RESET ROLE

Test Results

63 passed, 7 warnings in 0.61s

Pre-existing failures (not introduced by this PR):
test_oauth_connection_leak.py — 12 tests use pytest.mark.asyncio but pytest-asyncio is not installed. These tests document a real concern (authenticate_user() still uses a raw asyncpg.connect() for the login flow) that should be addressed separately.


Files Changed

50 files changed, 303 insertions(+), 461 deletions(-)
  • api/app/v1/endpoints/functions.py — set_role() rewrite
  • api/app/db/password_crud.py — DDL eradication
  • api/app/db/role_crud.py — DDL eradication
  • api/app/v1/endpoints/update/user.py — missed REVOKE/GRANT fix
  • api/app/v1/endpoints/create/policy.py — dead try/finally cleanup
  • api/app/v1/endpoints/update/policy.py — dead try/finally cleanup
  • api/app/v1/endpoints/{create,delete,update,read}/*.py (44 files) — RESET ROLE deletion
  • api/tests/test_*.py (5 files) — test alignment

Security Posture After This PR

  • Zero DDL privileges required by the ISTSOS_ADMIN service account for any user-management operation
  • Zero session-scoped role state can leak between requests via the connection pool
  • Zero individual PostgreSQL login roles are referenced anywhere in application code
  • The codebase is now architecturally ready for Week 3: adding set_config('app.user_id', ...) to set_role() for per-row RLS enforcement is a 2-line addition with zero structural changes required

Closes Issue istSOS#28 — eliminates two-step user provisioning.

After POST /Users, a newly created user had no RLS policy and could not
access any data until an administrator separately called POST /Policies.
This commit fixes that by automatically calling the appropriate policy
function inside the same transaction as user creation.

Changes:
- api/app/v1/endpoints/create/user.py
  * Add module-level _POLICY_FN_MAP (viewer/editor/obs_manager/sensor).
  * Capture app_role before get_db_role_for_rbac() remaps it, so the
    correct policy function can be dispatched.
  * After GRANT, call sensorthings.<role>_policy([username], policyname)
    with policyname = '{username}_default'. Administrator is skipped —
    admins bypass RLS by privilege, not by policy.
  * Policy functions already exist in the DB (istsos_auth.sql); no
    migration required.

- api/app/v1/endpoints/functions.py
  * Add docstrings to _validate_role_identifier() and set_role().

- api/app/v1/endpoints/create/data_array_observation.py
  * Import shared set_role() helper (was already using the correct
    upstream version; this import makes the dependency explicit).

- api/tests/test_rls_policy_creation.py (new)
  * Tests: correct policy function per role, administrator exclusion,
    naming convention, users_ as text[].

- api/tests/test_rbac_set_role_safety.py (new)
  * Tests: identifier validation, injection rejection, shared helper
    usage in data_array_observation.
- Add auth_provider and external_sub_id columns to sensorthings."User"
  via idempotent migration (001_identity_linking.sql) with a partial
  unique index on (auth_provider, external_sub_id) WHERE auth_provider
  IS NOT NULL, so local password users are completely unaffected.

- Introduce PENDING_ROLE sentinel in rbac_roles.py. The 'pending' state
  is intentionally absent from VALID_RBAC_ROLES so it can never be
  assigned through the public API; existing role validation is unchanged.

- Gate pending accounts in get_current_user() (oauth.py): after the DB
  lookup, any user with role='pending' immediately receives HTTP 403
  'Account pending admin activation' before any SET ROLE or handler
  body is reached.

- Add oidc_user_crud.py with create_pending_oidc_user() and
  get_user_by_provider_sub(). The insert function hardcodes role to
  PENDING_ROLE and contains zero DDL (no CREATE ROLE / CREATE USER),
  giving new OIDC accounts zero PostgreSQL footprint until activation.

- Add POST /Users/{id}/activate endpoint (activate_user.py), restricted
  to administrators. Runs UPDATE role, CREATE ROLE NOLOGIN IN ROLE,
  GRANT, and RLS policy assignment inside a single transaction so a
  failed step leaves the user still 'pending' with no partial state.

- Register activate_user router in api.py inside the AUTHORIZATION guard.

Local password users (POST /Users) are completely unaffected; no changes
were made to create/user.py.

Relates-to: GSoC 2026 Identity Linking architecture
- Add PasswordUpdateRequest Pydantic v2 schema (models/password.py)
  enforcing: min 12 chars, at least 1 uppercase, at least 1 digit.
  Violations surface as HTTP 422 before any DB is touched.

- Add update_local_password() CRUD function (db/password_crud.py):
  1. Fetch user row by ID → 404 if missing.
  2. OIDC guard: block auth_provider IS NOT NULL users with HTTP 400
     'External identities cannot update passwords locally'.
  3. Verify current_password via asyncpg.connect() (PostgreSQL auth layer)
     → 401 on InvalidPasswordError. No Python-side passlib used.
  4. Execute ALTER USER <username> WITH ENCRYPTED PASSWORD <new_password>
     using pg_quote_ident / pg_quote_literal to prevent injection.

- Add PATCH /Users/{id}/password endpoint (update/password.py):
  owner-or-admin guard; returns 204 No Content on success.

- Register update_password router in api.py inside AUTHORIZATION guard.

- Add test_password_update.py (9 tests, all pass, no live DB required):
  schema: valid, too-short, no-uppercase, no-digit
  crud: 404, 400 OIDC block, 401 wrong password, 204 ALTER USER issued
  endpoint: 403 non-owner/non-admin guard

Depends on: feat/identity-linking-jit-provisioning (requires auth_provider column)
…le-JWT fix

- Add RoleUpdateRequest Pydantic v2 schema (models/role.py):
  delegates to validate_rbac_role(); blocks 'administrator' (bootstrap-only)
  and 'pending' (internal state) with HTTP 422 before any DB is touched.
  Docstring explains the security boundary explicitly.

- Add update_user_role() CRUD function (db/role_crud.py):
  All mutations run inside a single asyncpg transaction (FOR UPDATE lock):
  1. 404 if user not found.
  2. 400 if user is in 'pending' waiting room.
  3. No-op early return if current_role == new_role (no DDL issued).
  4. 409 if demoting the last administrator (lockout guard).
  5. UPDATE sensorthings."User" SET role = new_role.
  6. REVOKE <old_pg_group_role> / GRANT <new_pg_group_role> only when the
     underlying PostgreSQL group role changes (e.g. viewer→obs_manager).
     viewer→editor shares the same 'user' PG role — no DDL issued.
  pg_quote_ident used for all identifier interpolation.

- Add PATCH /Users/{id}/role endpoint (update/role.py):
  administrator-only guard at router layer; returns 204 No Content.

- Register update_role router in api.py inside AUTHORIZATION guard.

- Add comment to get_current_user() in oauth.py documenting that role is
  fetched live from the DB on every request (not from the JWT payload),
  eliminating stale-JWT vulnerabilities after role changes. No logic change.

- Add test_role_reassignment.py (12 tests, all pass, no live DB needed):
  schema: valid, administrator/pending/unknown blocked
  crud: 404, 400 pending, no-op, 409 last-admin, REVOKE+GRANT, no-DDL
  endpoint: 403 non-admin guard

Depends on: feat/password-updates (stacked)
…dentials

Users are strictly application-level entities managed via sensorthings."User".
The backend connects to PostgreSQL through a single master service account;
individual users have no PostgreSQL login roles.

- Add shared POLICY_FN_MAP in rbac_roles.py as single source of truth for
  RLS policy dispatch (used by create/user.py and activate_user.py)
- Role reassignment (PATCH /Users/{id}/role) is a pure UPDATE on User.role;
  last-admin lockout locks all admin rows via SELECT … FOR UPDATE before
  counting to prevent concurrent demotion race condition
- User activation (POST /Users/{id}/activate) updates User.role and applies
  the corresponding RLS policy function — no PostgreSQL DDL
- User creation stores bcrypt hash in User.password via parameterised UPDATE

Removed: CREATE USER, CREATE ROLE, REVOKE, GRANT, ALTER USER DDL.

Refs: istSOS#190
BREAKING: set_role() now maps app-layer roles to PG group roles via
SET LOCAL ROLE instead of SET ROLE <username>. All 72 RESET ROLE
instances deleted across 41 endpoint files.

Refactor 1 — functions.py::set_role():
  - Maps viewer/editor → 'user', sensor/obs_manager → 'sensor', etc.
  - SET LOCAL ROLE (transaction-scoped, auto-reverts on COMMIT/ROLLBACK)
  - Removed inner connection.transaction() nested savepoint
  - Eliminated entire RESET ROLE call class (pool leak prevention)

Refactor 2 — password_crud.py:
  - Removed asyncpg.connect() credential verification
  - Removed ALTER USER … WITH ENCRYPTED PASSWORD DDL
  - Modern path: passlib/bcrypt verify + UPDATE User.password
  - Legacy fallback: get_auth_connection() for NULL password JIT migration

Refactor 3 — role_crud.py:
  - Removed REVOKE/GRANT DDL block (lines 167-196)
  - Role reassignment is pure UPDATE sensorthings."User" SET role
  - Removed _ADMIN_PG_ROLE, DB_ROLE_BY_RBAC_ROLE, pg_quote_ident imports

Refactor 4 — Test suite alignment:
  - test_set_role_sql_safety: asserts SET LOCAL ROLE + group role mapping
  - test_rbac_set_role_safety: parametrised 5 app roles → PG group roles
  - test_password_update: asserts UPDATE + bcrypt hash (not ALTER USER)
  - test_role_reassignment: asserts UPDATE only (not REVOKE/GRANT)
  - test_policy_role_switch: asserts SET LOCAL ROLE + zero RESET ROLE

49 files changed, 298 insertions(+), 442 deletions(-)
63/63 tests passing.
update/user.py lines 119-135 contained live REVOKE/GRANT statements
that were missed in the PR5 cleanup pass. These operated on individual
usernames as PostgreSQL role identifiers, which no longer exist under
the app-layer credential model.

Remove the entire DDL block. Role changes are reflected in the
sensorthings."User".role UPDATE above; set_role() maps the new role
to its PG group role dynamically at request time.
@KinshukSS2
KinshukSS2 force-pushed the fix/post-pivot-cleanup branch from b489eea to 75542cf Compare July 21, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant