fix/calendar stdlib shadowing - #254
Open
dbosk wants to merge 23 commits into
Open
Conversation
dbosk
commented
Nov 26, 2025
dbosk
commented
Owner
- Rename calendar module to canvas_calendar to avoid shadowing stdlib
This commit adds support for analyzing New Quizzes (also known as Quizzes.Next) in addition to Classic Quizzes. Both quiz types now work seamlessly with the `canvaslms quizzes analyse` command. Changes: - Add is_new_quiz() helper to detect New Quiz assignments - Add create_new_quiz_report() to generate reports via New Quiz API - Add poll_progress() shared helper for both quiz types - Add download_csv_report() shared helper for both quiz types - Refactor classic quiz code to use shared helpers - Update quiz detection logic to handle both Classic and New Quizzes - Update documentation to reflect New Quizzes support Both Classic Quizzes and New Quizzes now follow the same workflow: generate report → poll progress → download CSV → analyze data 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Fix create_new_quiz_report to pass parameters as kwargs (not _kwargs dict) - Use correct Canvas requester: _Canvas__requester (name-mangled) - Add poetry run testing instructions to CLAUDE.md Testing shows: - Classic quiz path works correctly - New Quiz API calls now reach the endpoint (no more AttributeError) - Getting 404 for New Quizzes (likely no submissions or API issue) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Testing Results: - Classic Quizzes: Work perfectly (tested with "Delutvärdering" survey) - New Quizzes: API returns 404 Not Found errors Documentation Updates: - Added note about New Quiz Reports API availability and limitations - Documented CSV export workaround for New Quizzes - Added troubleshooting section explaining potential causes: * API not enabled on all Canvas instances * Missing permissions or feature flags * Quizzes with no student submissions The implementation follows Canvas API documentation correctly. The 404 errors suggest the New Quiz Reports API (released January 2025) may not be fully rolled out or may require special configuration. CSV export method works for both Classic and New Quizzes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Critical bug fix: When both Classic and New Quiz versions exist with the same name, the code was finding New Quizzes first (which return 404), preventing Classic Quizzes from being analyzed. Changes: - Reversed search order: now searches Classic Quizzes FIRST - Only searches New Quizzes if no Classic Quiz is found - This ensures reliable API access since Classic Quiz reports work correctly Testing confirmed: - "Avslutande diagnostiskt prov (version 2023)" now works (was finding New Quiz ID 293507, now finds Classic Quiz ID 57165) - "Delutvärdering" survey continues to work perfectly - Both generate reports and analyze data successfully Rationale: Classic Quizzes have stable, working API support. New Quizzes may return 404 depending on Canvas configuration. By preferring Classic Quizzes, we maximize reliability while still supporting New Quizzes when no Classic version exists. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Major fixes to the `quizzes list` command:
1. **Eliminated duplicate listings**
- Classic Quizzes were appearing twice (once from get_quizzes(), once
as assignments with is_quiz_assignment=True)
- Now tracking quiz IDs to prevent duplicates
2. **Fixed New Quiz detection**
- Old method: checked is_quiz_assignment (caught Classic Quiz assignments)
- New method: uses get_new_quizzes() API which returns actual New Quizzes
- Updated is_new_quiz() to check object class (NewQuiz vs Quiz)
3. **Updated quiz search in analyse command**
- Now searches Classic Quizzes first (via get_quizzes())
- Then searches New Quizzes (via get_new_quizzes())
- Both paths use correct quiz type detection
Testing results:
- prgi24: No duplicates, finds "Datorprov (v2024)" as New Quiz ✓
- prgi25: No duplicates, finds "Exempelprov" as New Quiz ✓
- Classic Quiz analysis: Still works perfectly ✓
- New Quiz analysis: Correctly identifies New Quiz, attempts API (gets 404
as expected due to API availability) ✓
Note: Underscores in LaTeX code blocks should not be escaped with backslashes
when using [[...]] notation in noweb.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed incorrect underscore escaping in [[...]] blocks throughout quizzes.nw: - [[get_quizzes()]] (not [[get\_quizzes()]]) - [[get_new_quizzes()]] (not [[get\_new\_quizzes()]]) - [[_requester]] (not [[\_requester]]) - [[is_quiz_assignment]] (not [[is\_quiz\_assignment]]) - [[get_quiz_report]] (not [[get\_quiz\_report]]) - [[create_report()]] (not [[create\_report()]]) The [[...]] notation in noweb automatically handles escaping, so manual backslash escaping is incorrect and unnecessary. Added documentation to CLAUDE.md: - Noweb Escaping Rules section explaining proper [[...]] usage - Examples showing correct vs incorrect escaping - Clarifies that underscores only need escaping in regular LaTeX text, not inside [[...]] code references 8 instances corrected in quizzes.nw. Build and tests verified working. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add helper functions for processing long question titles: - extract_question_id(): Extract and remove question ID prefix - create_short_title(): Truncate at sentence boundaries or 80 chars - clean_newlines(): Convert \n to spaces for inline display - detect_and_format_code(): Format code snippets in blocks - escape_latex_complete(): Complete LaTeX special character escaping - Update question header output to use short titles - Display full question text in body when truncated or containing code - Remove question IDs from visible titles (use in LaTeX labels only) - Fix LaTeX compilation issues with newlines and special characters Tested with both New Quizzes (Exempelprov) and Classic Quizzes (Delutv). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Improvements to question title handling: - Add unescape_newlines() to convert literal \n strings from CSV - Update create_short_title() to detect code keywords and stop before code - Reorder processing: unescape first, detect code, then clean for display Response value display: - Remove [:70] truncation to show full option text - Use escape_latex_complete() for proper LaTeX escaping LaTeX output enhancements: - Add --standalone option to generate complete LaTeX documents - Add generate_latex_preamble/postamble() functions - Include verbatim and inputenc packages in preamble - Wrap output when standalone mode is enabled Code formatting: - Improve detect_and_format_code() to work with unescaped newlines - Add code block formatting for markdown and LaTeX Note: Code-on-one-line issue may require further investigation depending on actual CSV export format from Canvas. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Replace unescape_newlines() with comprehensive process_html_formatting(): - Converts <br> and <br/> tags to newlines - Extracts content from <pre> tags and marks as code - Unescapes HTML entities (<, >, &, , etc.) using html.unescape() - Strips inline HTML tags (<code>, <span>, <strong>, etc.) - Converts block elements (<p>, <div>) to newlines - Maintains backward compatibility with literal \n strings Update detect_and_format_code() to accept has_pre_tag parameter: - When Canvas marks content with <pre>, trust that signal - Check for code patterns and format entire text as code block - Improves code detection reliability over heuristics alone Update call site in <<output question header>>: - Call process_html_formatting() which returns (text, has_pre_tag) tuple - Pass has_pre_tag to detect_and_format_code() for better detection The user discovered via pdb debugging that Canvas CSV exports contain HTML formatting which varies by question. This implementation uses Python standard library (html.unescape() and re module) with no new dependencies. Tested with both New Quizzes (Exempelprov) and Classic Quizzes (Delutv). Note: Some questions may still show code on one line if Canvas stores them that way without newlines in the HTML. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit enhances the quiz analysis command to properly handle code formatting and HTML content from Canvas CSV exports: - Process HTML entities and tags in response values (both quantitative and qualitative), preventing HTML like <pre> from appearing in output - Add --use-minted option for syntax-highlighted code formatting - Implement single-line vs multi-line code detection, using appropriate LaTeX commands (\verb/\mintinline for single-line, verbatim/minted environments for multi-line) - Convert multiple consecutive spaces to newlines with preserved indentation, handling code that lacks proper <pre> tags - Update LaTeX preamble generation to support minted package These changes work together to ensure code snippets are properly formatted with correct line breaks and indentation, while HTML from Canvas is cleanly converted to plain text. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Changed --use-minted from a boolean flag to accept an optional language
argument for minted syntax highlighting. This allows specifying different
programming languages beyond the default.
Usage:
- --use-minted → Uses Python (default language)
- --use-minted bash → Uses Bash syntax highlighting
- --use-minted java → Uses Java syntax highlighting
- (no flag) → Uses verbatim (no syntax highlighting)
Implementation changes:
- Argument parser: Changed from action="store_true" to nargs='?'
with const="python" and default=False
- Renamed parameter: use_minted → minted_lang throughout codebase
- Updated detect_and_format_code() to use language value in
\begin{minted}{LANG} and \mintinline{LANG}{...}
- Updated documentation strings to reflect new parameter semantics
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Response values in quiz analysis were using incomplete escaping that only
handled 3 of 10 LaTeX special characters (_, %, &), causing compilation
failures when responses contained characters like #, $, {, }, ~, ^, or \.
Root cause: Two output locations (multi-select options at line 1320 and
qualitative responses at line 1411) used partial escaping instead of the
complete escape_latex_complete() function.
Changes:
- Line 1320: Replace partial escaping with escape_latex_complete(value)
- Line 1411: Replace partial escaping with escape_latex_complete(resp.replace('\n', ' '))
This makes all three response output locations use consistent, complete
escaping. The escape_latex_complete() function properly handles all 10
LaTeX special characters: \ & % $ # _ { } ~ ^
Tested with both New Quizzes and Classic Quizzes - LaTeX output now
compiles successfully with all special characters properly escaped.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Improved code detection to catch more Python code patterns that were being missed, particularly control flow statements and code with multiple consecutive spaces. Enhanced code pattern detection: - Added try/except blocks detection - Added elif/else statement detection - Added colon + indentation pattern (strong indicator of Python blocks) - Added assignment with type conversion (int, float, str, input, etc.) - Relaxed if/while patterns to not require comparison operators - Added print/input with string arguments - Added comparison operators and arithmetic operators - Added detection for multiple consecutive indented lines Enhanced code extraction logic: - Improved should_start_code heuristic with multiple indicators - Detects if/elif/else statements as code block starts - Detects lines ending with colon as block starts - Detects print/input statements as code indicators - Detects assignment to builtins/type conversions These patterns work together to catch code that lacks <pre> tags in Canvas CSV exports, particularly code formatted with multiple spaces instead of proper line breaks. Tested: Functions correctly detect and wrap code in verbatim blocks when tested in isolation. Further investigation needed for full integration in quiz analysis command. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
We want to output using the csv module, same as elsewhere. We can deduplicate code by combining Classic and New Quizzes fetching into a helper function.
Introduces filter_quizzes(), add_quiz_option(), and process_quiz_option() following the established process_*_option() pattern used throughout the codebase (courses, assignments, users modules). Changes: - Add filter_quizzes(course_list, regex): Searches both Classic and New Quizzes, yields matching Quiz objects with attached course reference - Add add_quiz_option() and process_quiz_option(): Provides consistent quiz selection interface with centralized error handling - Refactor list_command: Uses filter_quizzes() instead of inline loop, eliminates try/except wrapper, consistent with other list commands - Refactor analyse_command: Uses process_quiz_option() for unified quiz selection, switches to csv_files list pattern to support multiple quizzes, consolidates Canvas API report downloading into single chunk - Remove duplicate quiz search logic previously scattered in both commands Benefits: - Eliminates code duplication in quiz searching - Provides consistent EmptyListError handling across both commands - Enables future commands to reuse quiz selection logic - Follows established codebase patterns for consistency 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The local calendar.py module was shadowing Python's standard library calendar module, causing circular import errors when arrow library (a dependency) tried to import the stdlib calendar. Import chain that failed: - arrow library imports stdlib calendar - Python finds local calendar.py instead (due to sys.path ordering) - Local calendar.py imports canvasapi - canvasapi eventually needs requests - requests tries to import stdlib calendar.timegm - Python tries to get timegm from partially-initialized local calendar.py - ImportError due to circular import Changes: - Rename calendar.nw → canvas_calendar.nw - Update root chunk: <<calendar.py>> → <<canvas-calendar.py>> - Update imports in cli.nw: canvaslms.cli.calendar → canvaslms.cli.canvas_calendar - Update imports in discussions.nw: canvaslms.cli.calendar → canvaslms.cli.canvas_calendar - Add Makefile rules for two-stage tangling: - Stage 1: Tangle canvas_calendar.nw → canvas-calendar.py (hyphen for LaTeX safety) - Stage 2: Rename canvas-calendar.py → canvas_calendar.py (underscore for Python) - Add .INTERMEDIATE: canvas-calendar.py to clean up temporary file Benefits: - Eliminates stdlib module shadowing - Prevents circular import errors - Follows Python best practice of avoiding stdlib name collisions - Maintains LaTeX compatibility with hyphenated chunk names 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.