Skip to content

test(tools): add deterministic subsystem test suites for wikipedia, arxiv, and calculator tools - #301

Open
Jaswanth-dev-69 wants to merge 2 commits into
siddsachar:mainfrom
Jaswanth-dev-69:test/tool-subsystem-coverage
Open

test(tools): add deterministic subsystem test suites for wikipedia, arxiv, and calculator tools#301
Jaswanth-dev-69 wants to merge 2 commits into
siddsachar:mainfrom
Jaswanth-dev-69:test/tool-subsystem-coverage

Conversation

@Jaswanth-dev-69

Copy link
Copy Markdown

Summary

Adds deterministic subsystem test suites for three core tools that previously lacked subsystem test coverage: wikipedia_tool, arxiv_tool, and calculator_tool.

Changes Included

  • Wikipedia Tool (tests/subsystem/tools/test_wikipedia_tool_subsystem.py):
    • Identity (wikipedia), default enablement, and zero API key requirement contracts.
    • Automatic HTTP -> HTTPS API endpoint upgrade enforcement.
    • Defensive recovery behavior when upstream API returns JSON parse errors or network exceptions.
    • Document formatting and source metadata attribution (SOURCE_URL).
  • ArXiv Tool (tests/subsystem/tools/test_arxiv_tool_subsystem.py):
    • Search result formatting (Title, Authors, Published Date, Category, Abstract, HTML/PDF links).
    • Version suffix stripping for HTML links (https://arxiv.org/html/...).
    • Author truncation (et al. formatting when > 5 authors).
    • Empty search result and client failure handling.
  • Calculator Tool (tests/subsystem/tools/test_calculator_tool_subsystem.py):
    • Arithmetic operators, math functions (sqrt, log, sin, factorial, gcd, etc.), and constants (pi, e, tau).
    • Whole integer result formatting (no trailing .0).
    • Division by zero and invalid expression error boundaries.
    • Security validation: Verified that dangerous Python execution/imports (__import__, exec, eval, open, attribute access) are strictly blocked.
  • Source Test Map (tests/helpers/source_test_map.py):
    • Added tools_utility_and_retrieval rule so modifying any of the three tool source files automatically triggers these tests during CI test matrix runs.

Type of change

  • Bug fix
  • Feature
  • Docs
  • Refactor
  • Test-only change
  • Build / CI / release tooling

Risk area

  • Agent / prompts / tools
  • Designer
  • Memory / knowledge graph
  • Channels / external integrations
  • Installers / auto-update
  • UI only
  • Other

Testing

  • I ran uv run python scripts/run_test_matrix.py fast
  • I ran uv run python scripts/run_test_matrix.py pr for shared, release-sensitive, or cross-subsystem changes
  • I added or updated tests
  • I updated the relevant inventory in tests/helpers/ when changing coverage ownership
  • I manually tested the affected user flow
  • Not applicable, docs-only change

Release notes

  • User-visible change, release notes needed
  • Internal-only change, no release note needed

Checklist

  • Branch is based on latest main
  • No direct secrets, API keys, local paths, or private data included
  • The change is focused and does not include unrelated cleanup
  • Windows/macOS behavior considered where relevant

Copilot AI lite review requested due to automatic review settings August 5, 2026 17:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds deterministic subsystem coverage for three “utility/retrieval” tools (Wikipedia, arXiv, Calculator) and wires them into the source→test selection map so changes to the tool implementations automatically trigger these suites in the test matrix.

Changes:

  • Added new deterministic subsystem test suites for WikipediaTool, ArxivTool, and CalculatorTool, including formatting/error-boundary/security contracts.
  • Added a tools_utility_and_retrieval SourceTestRule to ensure tool-source edits select these tests in CI.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/subsystem/tools/test_wikipedia_tool_subsystem.py New deterministic subsystem tests for Wikipedia tool identity, HTTPS enforcement, error recovery, and output formatting.
tests/subsystem/tools/test_arxiv_tool_subsystem.py New deterministic subsystem tests for arXiv tool formatting, URL normalization, truncation, and error/empty handling.
tests/subsystem/tools/test_calculator_tool_subsystem.py New deterministic subsystem tests for calculator operations, formatting, error boundaries, and blocked-dangerous expressions.
tests/helpers/source_test_map.py Adds a new source→test mapping rule for the three tool implementations to select these suites when changed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/subsystem/tools/test_wikipedia_tool_subsystem.py Outdated
Comment thread tests/subsystem/tools/test_arxiv_tool_subsystem.py
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants