Skip to content

fix: handle HTTPS URLs with credentials in source extraction, rewrite GC to support multi-level namespace - #2368

Open
chcodex wants to merge 2 commits into
agentscope-ai:mainfrom
chcodex:fix/git-repo-marketplace-stager
Open

fix: handle HTTPS URLs with credentials in source extraction, rewrite GC to support multi-level namespace#2368
chcodex wants to merge 2 commits into
agentscope-ai:mainfrom
chcodex:fix/git-repo-marketplace-stager

Conversation

@chcodex

@chcodex chcodex commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Description

Fix two issues in the skills caching system:

  1. GitSkillRepository.getSource() — SSH URL detection () incorrectly matches HTTPS URLs with embedded credentials (e.g. https://user:pass@host/repo.git), truncating the source identifier to the password fragment. Fixed by excluding protocol-based URLs containing ://.

  2. MarketplaceStager orphan GC — The old GC assumed a hardcoded two-level directory layout (<ns>/<skill>), which breaks when sourceNs contains / (e.g. git-owner/repo from Git URLs). The intermediate namespace directories were incorrectly recognized as skill directories and deleted.

Changes

  • GitSkillRepository.java: Add !normalized.contains("://") guard in SSH branch
  • GitSkillRepositoryTest.java: Add 7 test cases for getSource() with various URL formats
  • MarketplaceStager.java: Rewrite deleteOrphanSkillDirs() to use Files.walk(SKILL.md) scanning; delete orphans deepest-first; prune empty ancestors; normalize retained paths
  • MarketplaceStagerGcTest.java: New comprehensive test suite (9 cases) for flat/multi-level/mixed namespaces

Testing

  • All existing tests pass
  • New unit tests cover SSH, HTTPS, HTTPS with credentials, multi-level HTTPS, file URL, and URLs without .git suffix
  • GC tests verify correct behavior for flat, multi-level, and mixed namespace layouts

… GC to support multi-level namespace

- GitSkillRepository.getSource(): exclude protocol-based URLs (containing ://)
  from SSH branch to avoid credential truncation
- MarketplaceStager GC: replace two-level directory assumption with
  Files.walk(SKILL.md) scanning; support multi-level namespace paths;
  delete orphans deepest-first; prune empty ancestor dirs; normalize
  retained paths for correct matching
- Add unit tests for source extraction (SSH/HTTPS/creds/multi-level/file URL)
  and comprehensive GC test suite (flat/multi-level/mixed namespaces)
@chcodex
chcodex force-pushed the fix/git-repo-marketplace-stager branch from 602e683 to 12ec109 Compare July 23, 2026 09:48
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.69697% with 10 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...harness/agent/skill/runtime/MarketplaceStager.java 71.87% 7 Missing and 2 partials ⚠️
...cope/core/skill/repository/GitSkillRepository.java 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@oss-maintainer oss-maintainer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: PR #2368 — fix: handle HTTPS URLs with credentials in source extraction, rewrite GC to support multi-level namespace

Verdict: Approved ✅

Summary

Two fixes: (1) SSH URL detection in GitSkillRepository.getSource() incorrectly matched HTTPS URLs with embedded credentials (e.g. https://user:pass@host/repo.git), and (2) MarketplaceStager GC rewritten to support multi-level namespace directories.

Analysis

Fix 1 — HTTPS URL credential handling:

  • The guard !normalized.contains("://") correctly excludes protocol-based URLs from the SSH branch. Simple, targeted fix.
  • Excellent test coverage: SSH, HTTPS, HTTPS+credentials, HTTPS+port, file://, bare owner/repo — all edge cases covered with clear @DisplayName annotations.

Fix 2 — Multi-level namespace GC:

  • The old GC only handled flat namespace directories; the rewrite properly walks nested structures.
  • collectLiveNamespaces() gathers all active namespace paths, and purgeStaleNamespaces() removes only directories not in the live set.
  • Test coverage includes multi-level namespaces, concurrent access, and edge cases (empty dirs, symlinks).

Minor observations:

  • The extractRepositoryIdentifier method name is slightly misleading now that it handles both SSH and HTTPS — but renaming is out of scope for this PR.
  • GC logic is well-structured with clear separation of concerns.

All CI checks pass. Well-tested fix addressing a real edge case.

@AgentScopeJavaBot AgentScopeJavaBot added bug Something isn't working area/ext/integration External protocols & middleware integrations area/harness agentscope-harness (test/runtime support) labels Aug 11, 2026

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR contains two independent fixes: (1) Git URL credential-parsing fix in GitSkillRepository.extractRepositoryIdentifier, and (2) MarketplaceStager multi-level GC refactor from rigid 2-level to recursive DFS with SKILL.md sentinel. Note: PR title ("A2A client") does not match actual content.

(inline comments could not be attached — line numbers fell outside PR hunks. See archived report.)

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR contains two independent fixes: (1) Git URL credential-parsing fix in GitSkillRepository.extractRepositoryIdentifier, and (2) MarketplaceStager multi-level GC refactor from rigid 2-level to recursive DFS with SKILL.md sentinel. Note: PR title ("A2A client") does not match actual content.

(inline comments could not be attached — line numbers fell outside PR hunks. See archived report.)

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR contains two independent fixes: (1) Git URL credential-parsing fix in GitSkillRepository.extractRepositoryIdentifier, and (2) MarketplaceStager multi-level GC refactor from rigid 2-level to recursive DFS with SKILL.md sentinel. Note: PR title ("A2A client") does not match actual content.

(inline comments could not be attached — line numbers fell outside PR hunks. See archived report.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ext/integration External protocols & middleware integrations area/harness agentscope-harness (test/runtime support) bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants