Skip to content

Fix % comment-stripping inside verbatim-family environments - #270

Open
rohith500 wants to merge 2 commits into
psi-oss:mainfrom
rohith500:fix/verbatim-comment-stripping
Open

Fix % comment-stripping inside verbatim-family environments#270
rohith500 wants to merge 2 commits into
psi-oss:mainfrom
rohith500:fix/verbatim-comment-stripping

Conversation

@rohith500

@rohith500 rohith500 commented Aug 9, 2026

Copy link
Copy Markdown

What broke

_visible_tex_content() treats % as a comment character everywhere, but inside verbatim, lstlisting, and minted environments, % is not a comment in real LaTeX — it's literal text. So a % inside a code listing was silently deleting everything after it from the manuscript text that gets checked for problems.

Why it's worse than it looks

Once the code hits a real closing \end{...} tag that it fails to recognize, the "I'm inside a listing" state never turns off — so every line for the rest of the document gets treated as a comment too, not just the one line. For example: a MATLAB code listing that uses % as its real comment character, containing something like \end{document} shown as sample text, would silently turn off citation and TODO checking for the entire rest of the paper.

Fix

_visible_tex_content() now tracks whether it is currently inside a verbatim-family environment, line by line, and stops stripping % while inside one. A separate check used for \begin{}/\end{} balance tracking also ignores literal LaTeX-looking text shown as example code inside a listing, so it doesn't get mistaken for a real environment marker.

Verification

  • Reproduced in isolation: confirmed \label{} and TODO markers were vanishing from the scanned text when placed inside a listing
  • Reproduced through the real pipeline: ran build_paper() with one real citation in the .bib file and one fake citation hidden inside a code listing — before the fix, the fake citation was never flagged as unresolved; after the fix, it correctly is
  • uv run pytest tests/ -q: 12,948 passed, 15 skipped, 0 failed
  • pre-commit run --all-files: clean
  • Addressed CodeRabbit's review feedback on this PR: fixed a related leak on the opening side of a listing, and corrected the CHANGELOG wording around finding severity

Checklist

  • CLA (signs automatically via CLA Assistant)
  • Tests added/updated
  • CHANGELOG.md updated under ## vNEXT

…/lstlisting/minted)

_visible_tex_content() treated % as a LaTeX comment unconditionally, but
verbatim-family environments don't parse comments -- % is literal text
there. This silently hid TODOs, \label{}, and \cite{} inside code listings
from validate_tex_draft() and check_citation_bib_coherence(). Worse: once
the closing-tag match failed to pop the verbatim state, everything after
it in the document went dark too, not just the one line.

Verified via isolated repro and a live build_paper() run with a real
citation in the bib and a fake one hidden inside a listing.
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The LaTeX validator now handles verbatim, lstlisting, and minted content separately from environment structure. It preserves findings inside listings, ignores literal environment markers, adds regression coverage, and documents the behavior.

Changes

Verbatim-aware LaTeX validation

Layer / File(s) Summary
Build verbatim-aware scan views
src/gpd/core/paper_quality.py
The scanner recognizes verbatim-family environments, preserves their body text for content checks, and excludes their markers from structural scanning.
Apply filtered views to validation
src/gpd/core/paper_quality.py
Draft validation uses separate views for placeholder, citation, label, equation, and environment-balance checks.
Cover verbatim validation behavior
tests/core/test_paper_quality.py, tests/test_citation_coherence.py, CHANGELOG.md
Regression tests cover comments, nested and same-line environments, matching closing tags, resumed linting, and citations in listings. The changelog records the behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the primary fix for percent-comment stripping inside verbatim-family environments.
Description check ✅ Passed The description explains the problem, motivation, implementation, verification, tests, and changelog update, with only minor template differences.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 7: Update the changelog entry describing validate_tex_draft() so hidden
\label{} content is said to affect label-validation findings, not
blocker-severity checks. Preserve the existing description of empty and
duplicate labels as major-severity findings.

In `@src/gpd/core/paper_quality.py`:
- Around line 869-877: The environment scan currently includes tokens after a
verbatim opener, causing false nested environment findings. In
src/gpd/core/paper_quality.py#L869-L877, update the logic around
_visible_tex_line and begin_match to retain full stripped as scan_text but use
stripped[:begin_match.end()] as env_scan_text when an opener is found; update
tests/core/test_paper_quality.py#L917-L931 to assert the result contains only
("placeholder_marker", 4).
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 334f99c3-e80d-41bc-843f-1c458afbc56c

📥 Commits

Reviewing files that changed from the base of the PR and between 0f41769 and 004555a.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • src/gpd/core/paper_quality.py
  • tests/core/test_paper_quality.py
  • tests/test_citation_coherence.py

Comment thread CHANGELOG.md Outdated
Comment thread src/gpd/core/paper_quality.py
…GELOG severity wording

<your one-line summary of what this does>

<paragraph: the begin-line bug CodeRabbit caught — text after a verbatim
opener on the same line was leaking into the env-balance view, so a
literal-lookalike \begin{lstlisting} appearing after a real
\begin{verbatim} on one line got treated as a genuine second open.
Same class of bug as the closing-side fix already in this PR, just
never mirrored to the opening side.>

<one line: CHANGELOG wording also corrected — \label{} findings are
major severity, not blocker, per CodeRabbit's note>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant