Skip to content

Reorganize Babel Validation for more extensibility#101

Merged
gaurav merged 8 commits into
mainfrom
split/1-library-reorg
Jun 26, 2026
Merged

Reorganize Babel Validation for more extensibility#101
gaurav merged 8 commits into
mainfrom
split/1-library-reorg

Conversation

@gaurav

@gaurav gaurav commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Pure-ish refactor, no service-behavior change. Moves the code under tests/common/ into an importable src/babel_validation package and updates the Google Sheet test modules to match.

What changes

  • Split the monolithic tests/common/google_sheet_test_cases.py into:
    • core/testrow.pyTestRow / TestStatus / TestResult
    • services/{nodenorm,nameres}.pyCachedNodeNorm / CachedNameRes
    • sources/google_sheets/{google_sheet_test_cases,blocklist}.py
  • Gsheet test modules now import from the new locations and parametrize lazily in pytest_generate_tests via tests/_pytest_helpers.deselected_by_markexpr, so marker-deselected runs never hit the network.
  • Migrate tests/pytest.ini into [tool.pytest.ini_options]; add a hatchling build that packages src/; add filelock (gsheet disk cache).
  • Move test_env.pytests/test_environment/.

Two incidental one-liners ride along in the final test files (NameRes biolink_type passed as a list; description identifiers as lists not sets) — kept as-is so #67 rebases cleanly.

Verify

  • uv run pytest -m unit --collect-only -q — full suite imports/collects offline (no network).
  • uv run pytest tests/test_environment/test_env.py — live Google Sheet download+parse through the moved code.

🤖 Generated with Claude Code

gaurav and others added 2 commits June 26, 2026 00:50
Move the timeout setting out of tests/pytest.ini into
[tool.pytest.ini_options], add testpaths=["tests"] so a bare pytest no
longer scans the website node_modules, and add a hatchling build-system
that packages src/ so `from src.babel_validation...` imports resolve when
installed. Add filelock, used by the Google Sheet disk cache.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate the code under tests/common into an importable
src/babel_validation package, splitting the monolithic
google_sheet_test_cases module into core (TestRow/TestStatus/TestResult),
services (CachedNodeNorm, CachedNameRes) and
sources/google_sheets (GoogleSheetTestCases, blocklist).

Update the Google Sheet test modules to import from the new locations and
to parametrize lazily in pytest_generate_tests via the new
tests/_pytest_helpers.deselected_by_markexpr, so marker-deselected runs
(e.g. `pytest -m unit`) never hit the network. conftest.py wipes the
Google Sheet disk cache at the start of each run. Move test_env.py into
tests/test_environment/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gaurav added a commit that referenced this pull request Jun 26, 2026
The repo's `.gitignore` predates the Python rewrite — it only covered
Scala/Giter8/IntelliJ artifacts, so `__pycache__/` and `*.pyc` showed up
as untracked noise throughout the tree and were easy to `git add` by
accident.

Appends the standard [GitHub
`Python.gitignore`](https://github.com/github/gitignore/blob/main/Python.gitignore)
template (bytecode, `build/`/`dist/`, `.pytest_cache/`, `.venv`/`.env`,
mypy/coverage caches, …) below the existing entries, plus a `.DS_Store`
line for macOS. Existing entries are kept at the top, so this is purely
additive.

Independent of the #67 split stack (#101#104) and based on `main`, so
it can merge immediately. No files are currently tracked that the new
patterns would retroactively ignore — `git ls-tree` shows no
`.pyc`/`__pycache__` in the tree — so nothing needs `git rm --cached`.

### Verify
```
git check-ignore -v src/__pycache__/x.pyc .venv .env   # all matched
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@gaurav
gaurav requested a review from Copilot June 26, 2026 17:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors shared Google Sheet–backed test utilities into an importable src/babel_validation library package, updates existing test modules to use the new import paths, and makes Google Sheet/blocklist parametrization lazy so marker-deselected runs avoid network access.

Changes:

  • Carves Google Sheet parsing + shared datatypes into src/babel_validation (core/services/sources) and updates tests to import from the new locations.
  • Adds lazy parametrization via pytest_generate_tests + tests/_pytest_helpers.deselected_by_markexpr to avoid collection-time network fetches for deselected runs.
  • Adds disk caching + file lock for Google Sheet CSV downloads; migrates pytest config into pyproject.toml and adds hatchling wheel build configuration.

Reviewed changes

Copilot reviewed 14 out of 23 changed files in this pull request and generated no comments.

Show a summary per file
File Description
uv.lock Adds filelock and updates lock metadata for editable local package.
pyproject.toml Adds filelock, hatchling build config, and moves pytest settings under [tool.pytest.ini_options].
tests/pytest.ini Removes legacy pytest.ini in favor of pyproject.toml configuration.
tests/conftest.py Deletes tempdir Google Sheet CSV cache on run start (controller-only) and keeps existing pytest configuration/fixtures.
tests/_pytest_helpers.py Adds helper to pre-evaluate -m mark expressions to skip network-backed parametrization when deselected.
tests/init.py Establishes tests as a package (enabling imports like tests._pytest_helpers).
tests/test_environment/test_env.py Updates import to new Google Sheet test-case module location.
tests/nodenorm/test_nodenorm_from_gsheet.py Switches to lazy Google Sheet parametrization and new import path.
tests/nodenorm/test_nodenorm_descriptions.py Changes description identifier collections from sets to lists.
tests/nameres/test_nameres_from_gsheet.py Switches to lazy Google Sheet parametrization, new import path, and sends biolink_type as a list.
tests/nameres/test_blocklist.py Switches blocklist loading to lazy parametrization and new import path.
src/init.py Establishes src as a package so src.babel_validation.* imports work.
src/babel_validation/init.py Initializes the new library package namespace.
src/babel_validation/core/init.py Initializes core subpackage.
src/babel_validation/core/testrow.py Introduces TestRow, TestStatus, and TestResult in the library.
src/babel_validation/services/init.py Initializes services subpackage.
src/babel_validation/services/nodenorm.py Adds CachedNodeNorm wrapper with in-process caching for NodeNorm calls.
src/babel_validation/services/nameres.py Adds CachedNameRes wrapper with in-process caching for NameRes calls.
src/babel_validation/sources/init.py Initializes sources subpackage.
src/babel_validation/sources/google_sheets/init.py Initializes Google Sheets source subpackage.
src/babel_validation/sources/google_sheets/google_sheet_test_cases.py Implements tempdir CSV cache + file lock and uses extracted TestRow.
src/babel_validation/sources/google_sheets/blocklist.py Adds blocklist Google Sheet loader + dataclass representation.
CLAUDE.md Updates architecture/docs to reflect new library layout and import paths.
Comments suppressed due to low confidence (2)

src/babel_validation/sources/google_sheets/google_sheet_test_cases.py:18

  • The library module is using an absolute import through the top-level src package. Using a relative import here avoids coupling internal modules to the packaging layout (and makes the planned rename away from src.babel_validation easier), while still working with the current structure.
    src/babel_validation/sources/google_sheets/google_sheet_test_cases.py:42
  • The cache filename uses only the first 8 hex chars of an MD5, which makes collisions plausible across different Google Sheet IDs; a collision would cause one sheet’s CSV to be reused for another sheet ID (incorrect test data). Using a longer/stronger hash (or the full digest) removes this risk with negligible downside.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gaurav and others added 3 commits June 26, 2026 13:52
- Switch absolute src.babel_validation import to a relative import (..core.testrow)
  so internal modules aren't coupled to the top-level packaging layout.
- Use full MD5 digest for cache filename instead of truncating to 8 chars,
  eliminating the theoretical hash-collision risk across different sheet IDs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gaurav gaurav changed the title Library reorg: carve shared test utilities into src/babel_validation (1/4) Reorganize Babel Validation for more extensibility Jun 26, 2026
gaurav and others added 3 commits June 26, 2026 14:22
Adds module docstrings explaining the per-(identifier, params) cache key,
the no-auto-eviction policy, and the intended cache-warming pattern:
call the batch method once for all identifiers in a task, then use the
single-item method per assertion at zero HTTP cost.

Adds method docstrings to from_url(), the batch methods (normalize_curies /
bulk_lookup), the single-item methods (normalize_curie / lookup), and the
cache-clearing methods.  Includes a note that lookup() targets a distinct
NameRes endpoint from bulk_lookup() and does not delegate to it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
NodeNormService and NameResService Protocols document the public interface
callers should type against.  When the implementation is later replaced by an
external library, any code typed against the Protocol will need no changes.

Rename clear_curie() → invalidate_curie() and delete_query() →
invalidate_query().  "Invalidate" is standard cache vocabulary and makes the
full-eviction-across-all-param-variants semantics clearer than "clear/delete".

Also tightens type annotations on normalize_curie / lookup signatures
(str, return type) to match the Protocol.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gaurav
gaurav merged commit a8c8188 into main Jun 26, 2026
@gaurav
gaurav deleted the split/1-library-reorg branch June 26, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants