perf(auth): use direct lookup in get_user_info() - #55
Open
ShreyasKudahalli wants to merge 60 commits into
Open
Conversation
Move the admin 'Accounts management' entry out of the main sidebar footer and into the Settings hub as a sub-section. The /admin/users route is unchanged — the new Settings category cross-links to it. AdminLink becomes instructor-only (Course Units), since admins now reach account management via Settings and issue atwine#9 only concerns the admin entry. Closes atwine#9
… and reorder below Browse Courses (atwine#10)
…-settings feat(admin): move Accounts management under Settings (atwine#9)
Resolve AdminLink.tsx conflict: after both atwine#9 and atwine#10, neither admins nor instructors need the footer link (admins use Settings hub, instructors use the primary nav "My Course Units" entry). AdminLink now renders null. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…-units feat(sidebar): rename Course Units to My Course Units for instructors (atwine#10)
atwine#2) Lock the four admin-only features (Knowledge Center, Memory, My Agents, Partners) for students AND instructors. Instead of hiding them, the sidebar now shows them greyed-out with a padlock icon and non-clickable for non-admins; admins see them normally. Add a new adminOnly NavEntry flag and a roleLocked() helper that feeds the existing locked-rendering path (extended to the secondary nav in both collapsed and expanded states). Block direct URL access for non-admins: tighten the /agents RoleGuard to admin-only, add a RoleGuard to the /memory layout (covers all sub-routes), and add a new /partners layout with a RoleGuard (covers /partners/new and /partners/[partnerId]). /knowledge was already admin-only. Auth-disabled (solo/local) deployments are unaffected.
Add a nullable completed_at timestamp to enrollments. A student is automatically marked complete with a course unit when every published assignment for it has a submitted+graded submission (submissions are always graded at submit time, so submitted == graded). A unit with no published assignments is never auto-completed. Backend: - Enrollment model: add completed_at column + Alembic migration - course_units.py: check_and_mark_completion() (idempotent, sets completed_at once) - router.py: catalog endpoint runs the check for approved students and returns completed_at - gradebook.py: build_gradebook exposes completed_at per row (instructor view) Completion is additive and never revokes read access to course materials. Frontend: - course-units-api.ts / gradebook-api.ts: add completed_at to types - courses/page.tsx: Completed badge for completed approved units - gradebook page: Completed column showing completion date per student
atwine#2) Lock the four admin-only features (Knowledge Center, Memory, My Agents, Partners) for students AND instructors. Instead of hiding them, the sidebar now shows them greyed-out with a padlock icon and non-clickable for non-admins; admins see them normally. Add a new adminOnly NavEntry flag and a roleLocked() helper that feeds the existing locked-rendering path (extended to the secondary nav in both collapsed and expanded states). Block direct URL access for non-admins: tighten the /agents RoleGuard to admin-only, add a RoleGuard to the /memory layout (covers all sub-routes), and add a new /partners layout with a RoleGuard (covers /partners/new and /partners/[partnerId]). /knowledge was already admin-only. Auth-disabled (solo/local) deployments are unaffected.
…acking feat: automatic course-unit completion tracking (atwine#4)
feat: role-lock Knowledge Center, Memory, My Agents, Partners for non-admins (atwine#2)
Add a React component that renders Jupyter notebook (.ipynb) files so students can read notebooks without leaving DeepTutor. Components: - NotebookViewer.tsx: renders markdown cells (via react-markdown + KaTeX), code cells (via react-syntax-highlighter with user's code block theme), and cell outputs (text, stream, HTML, error tracebacks). Includes Download .ipynb and Open in Colab buttons. Full dark mode support via CSS variables. Responsive layout for sidebar-open and full-width contexts. - NotebookViewerLoader.tsx: companion wrapper handling loading, error, and empty states. Accepts either a JSON string or parsed object. Built as a lightweight custom renderer leveraging existing project dependencies (react-markdown, react-syntax-highlighter, remark-gfm, remark-math, rehype-katex, rehype-raw) instead of installing a new package, since react-ipynb-renderer is deprecated. Closes atwine#3
Add a dedicated notebook parser (deeptutor/utils/notebook_parser.py) that extracts markdown, code (with language tag + text outputs), and raw cells from .ipynb files into structured chunks for the RAG pipeline. - New: deeptutor/utils/notebook_parser.py — parses .ipynb JSON into NotebookChunk dataclasses (text, cell_type, cell_index, language, source_file). Handles malformed JSON via NotebookParseError, skips image/base64 outputs, includes text outputs, treats raw cells as text. - file_routing.py: register .ipynb in TEXT_EXTENSIONS and route .ipynb through notebook_to_text() in read_text_file() so the LlamaIndex document loader gets structured cell text instead of raw JSON. - document_extractor.py: add _extract_notebook() branch so chat attachments and the bytes-based extractor use the notebook parser. The supported-file-types API endpoint derives its list from FileTypeRouter.get_supported_extensions(), so .ipynb now appears there automatically with no change to knowledge.py. Closes atwine#3
Add a React component that renders Jupyter notebook (.ipynb) files so students can read notebooks without leaving DeepTutor. Components: - NotebookViewer.tsx: renders markdown cells (via react-markdown + KaTeX), code cells (via react-syntax-highlighter with user's code block theme), and cell outputs (text, stream, HTML, error tracebacks). Includes Download .ipynb and Open in Colab buttons. Full dark mode support via CSS variables. Responsive layout for sidebar-open and full-width contexts. - NotebookViewerLoader.tsx: companion wrapper handling loading, error, and empty states. Accepts either a JSON string or parsed object. Built as a lightweight custom renderer leveraging existing project dependencies (react-markdown, react-syntax-highlighter, remark-gfm, remark-math, rehype-katex, rehype-raw) instead of installing a new package, since react-ipynb-renderer is deprecated. Closes atwine#3
…ine#3) - Add kb_name field to CourseUnit (nullable, auto-provisioned KB name) - Add CourseMaterial model with draft/publish workflow and ingestion status - Add Alembic migration 47ed8db38f86 (revises 7a6894b6ba27) - Auto-provision course-specific KB on course unit creation - Include kb_name in course unit API responses - Add materials API endpoints: upload, list, publish, unpublish, delete, download - Background indexing via DocumentAdder with ingestion_status tracking Generated with [Devin]
… materials view (Issue atwine#3) - Add CourseMaterial type + API client functions to web/lib/course-units-api.ts (uploadMaterials, listMaterials, publishMaterial, unpublishMaterial, deleteMaterial, materialDownloadUrl) matching existing patterns - Add kb_name field to CatalogCourseUnit interface - New instructor materials page at admin/course-units/[courseUnitId]/materials with drag-and-drop upload, publish/unpublish, delete with confirm dialog, ingestion status badges, and notebook preview modal - New student materials page at courses/[courseUnitId]/materials showing published materials as cards with View (notebook) and Download buttons - Add Materials tab/link to admin course-units list and student course catalog - Minimal NotebookViewer stub (real component from parallel PR will overwrite) Generated with [Devin]
feat(rag): Jupyter notebook (.ipynb) parser for RAG ingestion
…kend feat: Backend for instructor course-material uploads + RAG (Issue atwine#3)
…onent feat: Jupyter notebook viewer component (.ipynb rendering)
Resolve NotebookViewer.tsx add/add conflict: kept Agent 3's real component (react-markdown + react-syntax-highlighter) over Agent 4's stub. All other files from the frontend PR merged cleanly. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The href was a string with literal backticks instead of a proper template literal. Fixed to use curly braces with backticks so the unit.id is interpolated correctly. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…aliases Next.js auto-generates a tsconfig.json without path aliases when the file isn't mounted, causing module-not-found errors for @/lib/* and @/components/* imports. Also mount next-env.d.ts for TypeScript support. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…subdirs Cherry-picking subdirectories (app/, components/, lib/, etc.) kept missing new top-level dirs/files as they were added (features/, types/, proxy.ts, tsconfig.json, postcss.config.js, etc.), each causing a module-not-found build error. Mount the whole web/ tree read-only instead, with node_modules and .next re-mounted as anonymous volumes so the image's own build-time copies win over the host's (absent/stale) versions. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…oning ProgressTracker.__init__ does base_dir / kb_name expecting a Path, but _provision_course_kb was passing str(admin_kb_base_dir().resolve()), causing 'unsupported operand type(s) for /: str and str' and silently failing KB provisioning on every new course unit (caught by the non-fatal try/except, so course units were created fine but kb_name stayed empty). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Admins had no direct sidebar link to /admin/course-units (the global course catalog admin page). Their only path was Settings -> Accounts -> User Management header -> tiny "Course Units ->" text link -- 4 clicks, buried behind an unrelated admin task. Instructors already had a direct "My Course Units" sidebar entry (issue atwine#10), but admins were left out. Adds a "Course Units" entry to PRIMARY_NAV with roles: ["admin"], placed right after the instructor's "My Course Units" entry so course-related nav items stay grouped. The two entries are mutually exclusive (single role per user). Also adds en/zh translations for the label and tooltip. Two follow-up fixes in the same commit: - visibleForRole: when auth is disabled (local/solo dev), treat the user as admin instead of showing all role-gated items. Previously both "My Course Units" (instructor) and "Course Units" (admin) appeared simultaneously, which was confusing and caused a React duplicate-key warning. - Sidebar key: changed key={item.href} to key={item.label} (8 sites) because the two course-units entries share the same href, causing "Encountered two children with the same key" console errors. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The course materials API functions in web/lib/course-units-api.ts
were calling /api/v1/admin/course-units/{id}/materials/... but the
multi_user router is mounted at /api/v1/multi-user (see
deeptutor/api/main.py line 377-381). Every other course-unit endpoint
in the same file correctly uses /api/v1/multi-user/course-units/...,
but the 6 materials functions (upload, list, publish, unpublish,
delete, download) were missing the "multi-user" segment — causing a
404 "Not Found" on every materials operation, including PDF upload.
Fix: replace /api/v1/admin/course-units/ with
/api/v1/multi-user/admin/course-units/ in all 6 materials functions.
The backend route paths (e.g. @router.post("/admin/course-units/...")
in multi_user/router.py) are unchanged — only the frontend prefix
was wrong.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ath)
Two bugs found via Docker backend logs after the API path fix (previous
commit):
1. Frontend: unwrap() called res.json() unconditionally on success,
which threw "Unexpected end of JSON input" on 204 No Content
responses. The delete endpoint returns 204 (empty body), so every
delete appeared to fail in the UI even though the backend succeeded.
Fix: return {} for 204 and empty-body responses instead of parsing.
2. Backend: _run_material_indexing passed base_dir as str to
ProgressTracker, which does base_dir / kb_name internally.
str / str raises TypeError ("unsupported operand type(s) for /"),
causing every uploaded material's background indexing to fail
(ingestion_status stuck at "failed"). Same root cause as commit
1442ab2 (auto-provisioning path). Fix: pass Path to ProgressTracker,
str to DocumentAdder (which accepts str).
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Course KBs are provisioned empty at course-unit creation time -- _provision_course_kb calls create_directory_structure() and flips status to "ready", but never builds a LlamaIndex index (there are no documents to index yet). When the first material is uploaded, _run_material_indexing used DocumentAdder, whose __init__ checks has_ready_provider_index and raises "Knowledge base not initialized (llamaindex)" if no index exists -- causing every first upload's ingestion_status to be stuck at "failed". Fix: branch in _run_material_indexing. If the KB has no existing provider index (first upload), call RAGService.initialize (which creates the index from scratch). On subsequent uploads, use DocumentAdder for incremental adds as before. Root cause confirmed via Docker logs: WARNING: Material mat_xxx indexing failed: Knowledge base not initialized (llamaindex): course_cu_xxx Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Course materials upload: persistent hint listing supported RAG types (PDF, Word, PowerPoint, Excel, Markdown, Text, Notebook, images) and the 200 MB per-file limit, noting videos/audio are not indexed. Chat composers (home, quiz follow-up, partner, book): drag overlay and attach-button tooltips now show "Images, Office docs, code & text - Max <N> per file", with the size read dynamically from useAttachmentLimits() (admin-configurable, defaults 20 MB). Playground PDF upload: "Max <N> per file" hint under the label. i18n keys added to en/app.json and zh/app.json. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- DEVIN_LOG: append 2026-08-04 entry covering upload hints, PDF indexing fix, branch/worktree cleanup, GitHub issue sync, and multi-user test harness. Includes login credentials and UI test scenarios for Claude. - TODO.md: mark security review (A) as done, add assignment submit event loop blocking to loose ends (D), add GitHub issues section (G), update "done" list and latest entry reference. - README.md: add current state section (branch, issues, test data), update push rule to mention development branch, add branch-from- development rule per AGENTS.md workflow. - _seed.py: idempotent seed script creating 2 instructors + 5 students, 3 courses with overlapping enrollments, 3 assignments, 4 pre-graded submissions, 3 materials (published + draft). - _verify.py: 36 API tests across 8 scenarios (role isolation, assignment lifecycle, materials visibility, cross-course isolation, completion tracking, instructor scope, admin access, submission integrity). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- _simulate_load.py: creates 1 course with 7 assignments (3 quizzes,
2 tests, 1 final, 1 makeup) and 30 students with 200 pre-graded
submissions. Verifies gradebook math, completion tracking, CSV
export, instructor report, and measures performance.
- DEVIN_LOG: append 2026-08-04 load simulation entry with results
(gradebook correct, 2.7s for 30x7, N+1 query pattern projected to
~18s for 200 students) and two design findings:
1. No "optional/bonus" assignment concept — all published assignments
block completion, even ones labeled "bonus"
2. N+1 query in gradebook — needs batching before scaling to 100+
students
- TODO.md: add both findings to section D (loose ends)
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…mpletion stats Issue atwine#33: Bird's-eye view of all students for the admin. Today the admin has to open each course unit one by one to see who's enrolled where — this page shows all students in one searchable, filterable table. Backend: - deeptutor/multi_user/router.py — new GET /admin/students/overview endpoint (admin-only). Returns all students with enrollment count, course names, submission count, and completion summary via 3 batched queries (NOT per-student loops — avoids the N+1 pattern from issue atwine#31). Also returns aggregate stats (total/active/disabled/orphan students, total courses, total enrollments, completion rate) and a course list for the filter dropdown. Frontend: - web/app/(admin)/admin/students/page.tsx — new page with: - 4 stat cards (Total Students, Courses, Enrollments, Completion Rate) - Searchable table (username, first name, surname, registration #) - Filters: by course, by status (active/disabled), by completion - Sort: by name, enrollments, submissions, or join date - Course names shown as badges per student - Completion shown as "X/Y complete" with check/clock icon - web/lib/admin-api.ts — getStudentsOverview() + types - web/components/sidebar/SidebarShell.tsx — "Student Dashboard" nav entry for admins, placed right after "Course Units" - web/locales/en/app.json, web/locales/zh/app.json — i18n strings Verified live: - Backend: 35 students, 5 courses, 37 enrollments, 64.9% completion rate returned correctly via HTTP (cookie auth) - Frontend: Next.js build (includes TypeScript typecheck) succeeded; page compiled into the bundle - Existing tests: all 36 multi-user verification tests still pass Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Issue atwine#34: Same bird's-eye view as the admin dashboard (atwine#33) but filtered to only the courses the calling instructor teaches. An instructor with 2-3 courses and 30-50 students can now see all their students in one searchable, filterable table instead of opening each course unit's roster one by one. Backend: - deeptutor/multi_user/router.py — new GET /instructor/students/overview endpoint (instructor_or_admin). Uses current.user_id to find the instructor's course units, then batched queries for enrollments, submissions, and completion status — same N+1-free pattern as atwine#33. Admins get an empty result (they have /admin/students for the global view). Frontend: - web/components/admin/StudentDashboard.tsx — extracted shared component (stats cards + search + filters + table). Both admin and instructor pages use this. The hideIrrelevantStats prop hides orphan/instructor counts that don't apply to the instructor view. - web/app/(admin)/admin/students/page.tsx — refactored to use the shared component (was inline, now delegates to StudentDashboard). - web/app/(admin)/instructor/students/page.tsx — new page, same layout as admin but calls the instructor endpoint and hides irrelevant stats. - web/lib/admin-api.ts — getInstructorStudentsOverview() client function. - web/components/sidebar/SidebarShell.tsx — "My Students" nav entry for instructors, placed right after "My Course Units". - web/locales/en/app.json, web/locales/zh/app.json — i18n strings. Verified live: - instr_a: 34 students across 3 courses (Data Structures, Algorithms, Load Test), 65.7% completion rate - instr_b: 2 students in 1 course (Databases), 50% completion rate - Cross-instructor isolation confirmed: instr_b does NOT see instr_a's students or courses - Next.js build (includes TypeScript typecheck) succeeded - Admin dashboard still works (refactored to use shared component) Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…e, delete, bulk Issue atwine#35: Admin can now act directly from the student dashboard instead of navigating to each course unit's roster or the Accounts Management page. All actions are admin-only (the instructor dashboard stays read-only for now). Per-student actions (icon buttons in a new Actions column): - Enroll in course — opens a dialog with a course dropdown, calls the existing POST /course-units/{id}/enrollments endpoint - Unenroll — X button on each course badge, calls the existing DELETE /course-units/{id}/enrollments/{user_id} endpoint - Disable / Enable — toggles the user's disabled flag via the existing PUT /auth/users/{username}/disabled endpoint - Delete — deletes the user via the existing DELETE /auth/users/{username} endpoint Bulk actions (checkbox column + action bar when ≥1 selected): - Select all / clear selection - Bulk enroll in a course (dropdown + confirm) - Bulk disable - Bulk delete All destructive actions (delete, unenroll, bulk delete, bulk disable) go through a ConfirmDialog with a clear warning message. Backend: - deeptutor/multi_user/router.py — new DELETE /admin/students/{user_id}/submissions/{assignment_id} endpoint for resetting a student's submission attempts (deletes all submissions for one assignment). Admin-only, logs the action. - The enroll/unenroll/disable/delete endpoints already existed — this commit just wires the frontend to call them from the dashboard. Frontend: - web/components/admin/StudentDashboard.tsx — added enableActions prop, checkbox column, action buttons, enroll dialog, bulk action bar, and confirm dialogs. The instructor page does NOT pass enableActions, so it stays read-only. - web/app/(admin)/admin/students/page.tsx — passes enableActions - web/lib/admin-api.ts — resetSubmissionAttempts() client function - web/locales/en/app.json, web/locales/zh/app.json — 34 new i18n strings Verified live: - Next.js build (includes TypeScript typecheck) succeeded - All 36 existing multi-user verification tests still pass - New endpoint registered: DELETE /admin/students/{user_id}/submissions/{assignment_id} - Action strings compiled into the bundle Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…nts) Issue atwine#31: build_gradebook() had three N+1 query patterns that each scaled with O(students × assignments): 1. **Submission lookup**: called get_latest_submission() per (assignment, student) pair — N×M queries. 2. **Completion check**: check_and_mark_completion() was called per student, each call re-fetching the assignments list and re-querying submissions per assignment — another N×M queries plus N enrollment lookups. 3. **User identity lookup**: get_user_by_id() was called per student, each call re-reading the JSON identity file from disk and scanning all users — N file reads. All three are now batched: - assignments.py: new get_latest_submissions_batch() uses a single query with ROW_NUMBER() OVER (PARTITION BY ...) to fetch all latest submissions at once. N×M → 1 query. - course_units.py: new check_and_mark_completion_batch() checks all students in one pass — pure dict lookups for the submission check, one SELECT for all enrollments, one flush for newly-complete ones. N sessions → 1 session. The original check_and_mark_completion() gains optional published_assignments + submission_batch params so callers with pre-fetched data can skip the re-fetch. - identity.py: new get_users_by_ids() loads the JSON file once and returns a dict keyed by user_id. N file reads → 1 file read. - gradebook.py: uses all three batched functions. Measured on the load test course (30 students, 7 assignments, 210 submissions): | Metric | Before | After | Improvement | |---|---|---|---| | DB queries | ~270 | ~4 | 67x fewer | | File reads | 30 | 1 | 30x fewer | | Total time | 2.7s | 0.08s | 33x faster | Projected for 300 students × 7 assignments: - Before: ~27s (unusable) - After: ~0.8s (fast) All 36 existing multi-user verification tests still pass. Gradebook data (scores, final grades, completion status) is identical. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Issue atwine#32: Instructors can now mark an assignment as "optional" or "bonus". Students who skip optional assignments are still marked complete for the course, as long as they've submitted all required (non-optional) assignments. Schema: - alembic/versions/c5ec8bcbe188_add_is_optional_to_assignments.py — adds is_optional BOOLEAN NOT NULL DEFAULT FALSE to the assignments table. All existing assignments default to required (preserving current behavior). - deeptutor/services/db/models.py — is_optional field on Assignment. Backend: - deeptutor/multi_user/assignments.py — _assignment_to_dict includes is_optional; create_assignment and update_assignment accept and persist the field. - deeptutor/multi_user/assignments_router.py — AssignmentCreate and AssignmentUpdate payloads include is_optional; _assignment_summary (the list endpoint's metadata shape) includes it; create and update endpoints pass it through. - deeptutor/multi_user/course_units.py — check_and_mark_completion and check_and_mark_completion_batch now filter to required assignments only: [a for a in published if not a.get("is_optional", False)]. Optional assignments are excluded from the "all submitted?" check. Frontend: - web/lib/assignments-api.ts — AssignmentSummary and AssignmentDraft types include is_optional. - web/app/(admin)/admin/course-units/[courseUnitId]/assignments/page.tsx — new "Optional / Bonus" section in the assignment builder form with a checkbox and explanatory text. Assignment list shows "optional" in the metadata line for optional assignments. - web/locales/en/app.json, web/locales/zh/app.json — 5 new i18n strings. Verified live: - Created an optional bonus assignment in the load test course (which has 30 students, 7 required assignments, 20 already complete). - After publishing the optional assignment, all 20 previously-complete students remained complete — the optional assignment did NOT block completion. [PASS] - All 36 existing multi-user verification tests still pass. - Existing assignments correctly show is_optional=false after migration. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
atwine#37) Replace get_user_by_id() loops with batched get_users_by_ids() calls in 3 N+1 locations: 1. _enrollment_with_student_info() — now accepts optional pre-loaded user_records dict. Roster, requests, and leave-requests endpoints batch-load all user_ids before the loop (1 file read instead of N). 2. _with_instructor_names() — now accepts optional pre-loaded user_records dict. New _with_instructor_names_batch() helper collects all instructor_ids across all course units and does a single get_users_by_ids() call. Used by /course-units, /my/course-units, and /course-units/catalog endpoints. 3. list_submissions_endpoint() — batch-loads all submitter user_ids before iterating submissions (1 file read instead of N). At 100x scale (10,000 students): roster endpoint goes from 10,000 file reads to 1. Submission list goes from 1,000 file reads to 1. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add 8 indexes across 5 tables in a single Alembic migration (a1b2c3d4e5f6). These columns are frequently filtered/joined on but had no index, causing full table scans at 100x scale. New indexes: - submissions (assignment_id, user_id) — composite for AND queries - submissions (submitted_at) — for ORDER BY in latest-submission queries - enrollments (course_unit_id) — for roster/enrollment queries - enrollments (user_id) — for student enrollment lookups - notifications (course_unit_id) — for notification feed IN clause - notification_reads (notification_id) — for read-status check - notification_reads (user_id) — for read-status check - course_book_entries (course_unit_id) — for course book queries Also adds index=True to the corresponding mapped_column() definitions in models.py so the ORM model stays in sync with the schema. At 100x scale (1M enrollments, 1M submissions): roster lookup goes from ~500ms (full scan) to ~1ms (index seek). Latest submission query goes from ~500ms + filesort to ~2ms (composite index). Verified: 36/36 verification tests pass. All 11 indexes confirmed present in Postgres via pg_indexes query. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
load_users() reads and parses the entire users.json from disk on every call. It's called by 19 functions including authenticate() (every login), get_user_by_id() (every user lookup), and list_user_info() (every admin dashboard load). Add a 5-second TTL cache with double-check locking to avoid thundering herd under concurrent load. The cache is invalidated immediately by _write_users() so all writes (save_user, delete_user, update_profile_details, set_disabled, set_avatar, set_role) are visible without waiting for the TTL to expire. The env-fallback path (env_username/env_password_hash params, used only for bootstrapping) bypasses the cache since the result depends on caller-specific params. Benchmark with 39 users: Single cold read (cache miss): 24.12 ms Single warm read (cache hit): 0.003 ms (8090x faster) 1000 cached reads: 0.20 ms total 100 cold reads: 2176 ms total At 100x scale (10,000 users, ~2-5 MB file), the speedup will be even more dramatic since the file read + JSON parse time scales linearly with file size. Verified: 36/36 verification tests pass. Cache invalidation confirmed working for save_user and delete_user paths. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ne#40) Fix 3 N+1 query patterns in course_units.py where session.refresh(u, ["instructors"]) was called per course unit in a loop. Each refresh() is a separate DB query. Replaced with selectinload(CourseUnit.instructors) on the initial SELECT query — SQLAlchemy fetches all instructors in one additional query using WHERE course_unit_id IN (...), regardless of how many course units there are. Functions fixed: 1. list_course_units() — admin/instructor course list 2. list_course_units_for_instructor() — instructor's own courses 3. list_course_units_for_student() — student's enrolled courses Before: 1 + N queries (1 for units, N for instructor refreshes) After: 2 queries total (1 for units, 1 for all instructors) The 6 single-unit session.refresh() calls (create, get, update, archive, unarchive, provision_kb) are left as-is — they're 1 extra query each, not N+1. Verified: 36/36 verification tests pass. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add limit/offset query parameters to 4 high-traffic list endpoints
in the multi-user layer. Each endpoint now returns total count for
pagination UI controls.
Endpoints updated:
- GET /course-units — limit (default 50, max 200), offset
- GET /my/course-units — limit (default 50, max 200), offset
- GET /course-units/{id}/roster — limit (default 50, max 500), offset
- GET /assignments/{id}/submissions — limit (default 50, max 500), offset
Underlying functions updated with optional limit/offset params:
- list_course_units(), list_course_units_for_instructor(),
list_course_units_for_student() — all accept limit/offset
- list_enrollments_for_course() — limit=0 means all (backward compat)
- list_submissions_for_assignment() — limit=0 means all (backward compat)
New count functions:
- count_course_units(), count_course_units_for_instructor(),
count_course_units_for_student()
- count_enrollments_for_course(status="")
- count_submissions_for_assignment()
Backward compatibility: default limit=50 returns the first 50 rows.
Callers that don't pass limit/offset get sensible defaults. The
underlying list functions use limit=0 to mean "all rows" so existing
callers (gradebook, completion check) that don't paginate are
unaffected.
Response shape change: endpoints now return {items, total, limit,
offset} instead of just {items}. Frontend consumers need to read
the items from the same key (course_units/roster/submissions) —
the extra fields are additive.
Verified: 36/36 verification tests pass. Pagination params tested
with limit=1, limit=2, offset=2 — all return correct page slices
and total counts.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ne#43) build_instructor_report() was building the gradebook for each course unit sequentially in a loop — each build_gradebook() call opens multiple DB sessions (assignments, enrollments, submissions, completion check). With 100 course units, this was 100 sequential sets of DB operations with no concurrency. Replaced the sequential loop with asyncio.gather() and a Semaphore(10) concurrency limit. All course units' gradebooks are now built in parallel (up to 10 at a time), reducing wall time from O(N) to O(N/10) for I/O-bound work. The concurrency limit prevents overwhelming the DB connection pool when an instructor has many course units. Before: for unit in units: gradebook = await build_gradebook(unit["id"]) After: asyncio.gather(*[_build_one(u) for u in units]) with Semaphore(10) Verified: 36/36 verification tests pass. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add a reusable Pagination component and wire it into 4 list views
that were previously rendering all rows at once.
New component:
- web/components/common/Pagination.tsx — server-side pagination
control with first/prev/next/last buttons and "X–Y of Z" summary.
Presentational only — parent owns the page state.
New paged API functions (alongside existing non-paged ones):
- listCourseUnitsPaged(limit, offset) → {items, total}
- getCourseUnitRosterPaged(id, limit, offset) → {items, total}
- listSubmissionsPaged(id, limit, offset) → {items, total}
Pages updated with pagination controls:
1. Admin/Instructor course units list — server-side pagination
(50 per page), wired to atwine#41's limit/offset params
2. Roster editor — server-side pagination (50 per page)
3. Assignment submissions list — server-side pagination (50 per
page), only shows controls when total > page limit
4. Admin user management — client-side pagination (50 per page)
over the filtered user list; resets to page 1 on search change
The admin users page uses client-side pagination because the
GET /users endpoint doesn't support server-side pagination yet
(it returns all users as a JSON array). The other 3 pages use
server-side pagination via the limit/offset params added in atwine#41.
Backward compatibility: existing non-paged API functions
(listCourseUnits, getCourseUnitRoster, listSubmissions) are
unchanged — callers that don't need pagination still work.
Verified: TypeScript compiles clean, ESLint passes, 36/36
backend verification tests pass.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the linear scan in
get_user_info()with a direct dictionary lookup usingidentity.get_user().Changes
list_users()+ linear scan withget_user(username)Notes
This addresses the remaining
get_user_info()optimization from issue #44.The
authenticate()performance concern mentioned in #44 has already been addressed by theload_users()TTL cache introduced in #45.