perf(grades): optimize database queries for large-scale grade recalcu…

[andrey-canon]

Description

This PR addresses significant platform performance degradation caused by inefficient database queries during large-scale course grade recalculations.

The previous implementation relied on SQL OFFSET pagination, which forced the database to perform full index scans and discard hundreds of thousands of rows for high-offset tasks. Additionally, it suffered from an $N+1$ query problem by fetching user data individually for every enrollment in a batch.

Changes

• Keyset Pagination: Refactored _course_task_args and compute_grades_for_course to use start_id (ID-based seeking) instead of offset. This ensures O(1) database lookup performance regardless of the course size.
• Database Optimization: Replaced order_by('created') with order_by('id') to leverage the Primary Key clustered index.
• Eager Loading: Added .select_related('user') to the enrollment QuerySet to fetch user data in a single JOIN query, eliminating $100$ extra queries per batch.
• Memory Efficiency: Used .values_list('id', flat=True) in the task generator to minimize memory footprint when handling courses with 400k+ enrollments.

How to Test

Run the following script in the Django shell (python manage.py lms shell) on a high-enrollment course:

from common.djangoapps.student.models import CourseEnrollment
from opaque_keys.edx.keys import CourseKey
import time
from django.db import connection, reset_queries

course_key = CourseKey.from_string("your/course/id")
batch = 100
offset_test = 440000 

# Benchmark Legacy Logic
reset_queries()
st = time.time()
enrollments_legacy = CourseEnrollment.objects.filter(course_id=course_key).order_by('created')[offset_test:offset_test + batch]
ids_legacy = [e.user.id for e in enrollments_legacy]
print(f"Legacy Time: {time.time() - st:.4f}s | Queries: {len(connection.queries)}")

# Benchmark Optimized Logic
start_id = CourseEnrollment.objects.filter(course_id=course_key).order_by('id')[offset_test].id
reset_queries()
st = time.time()
enrollments_new = CourseEnrollment.objects.filter(course_id=course_key, id__gte=start_id).select_related('user').order_by('id')[:batch]
ids_new = [e.user.id for e in enrollments_new]
print(f"Optimized Time: {time.time() - st:.4f}s | Queries: {len(connection.queries)}")

Performance Benchmarks

Metric	Original (Offset + N+1)	Optimized (Seek + Join)	Improvement
Execution Time	~0.9614s	~0.0388s	~25x faster
DB Queries	101	1	100 fewer queries

Issue # 1744
Migration pr of #73

[johanseto]

I didnt understood the last column order of the results in the PR review. However with the other 2 is ok. Seems working.