Add inline-script cache key hash utility (PEP 723 PR 1/16) by StellaHuang95 · Pull Request #1634 · microsoft/vscode-python-environments

StellaHuang95 · 2026-07-14T21:01:38Z

Part of #1602 (PEP 723 inline script env support). Design doc: #1601.

Roadmap context

This is PR 1 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.

Phase	PR	Status
Phase 1: Foundation	PR 1: cache key hash utility	this PR
	PR 2: cache layout + `meta.json` sidecar	in progress
	PR 3: `requires-python` to interpreter selection	in progress
Phase 2: Manager	PR 4: `InlineScriptEnvManager` skeleton	merged (#1610)
	PR 5: `create()` happy path	not started (needs 1, 2, 3, 4)
	PR 6: `create()` uv-install fallback	not started (needs 3, 5)
	PR 7: persistence with `get`, `set`, and Memento	in progress (needs 4)
	PR 8: activation-time discovery	not started (needs 2, 4, 7)
Phase 3: Routing	PR 9: route PEP 723 scripts to the inline manager	not started (needs 4, 7)
	PR 10: per-script project registration	not started (needs 9)
Phase 3.5: Cross-repo	PRs 17-19: Pylance and vscode-python integration	not started
Phase 4: UX	PR 11: picker item; PR 12: bulk command	not started
Phase 5: Lifecycle and polish	PR 13: clear cache; PR 14: TTL; PR 15: telemetry; PR 16: status bar	not started

PRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path.

Why this PR

The cache key decides whether a PEP 723 script reuses an existing environment or builds a fresh one. Under the dependency-keyed design from Q4 of #1601, scripts using the same dependencies and interpreter should share an environment.

Trivial metadata edits such as changing package-name casing, separator style, whitespace, dependency order, or extras order should not fragment the cache.

This PR isolates that logic as pure functions with no filesystem access, global storage dependency, or extension activation behavior. The on-disk cache layout remains separate in PR 2.

What this PR does

Adds src/common/inlineScriptCacheKey.ts with four exports:

normalizeDependency(dep)

Canonicalizes common variants of the same requirement:
- Applies PEP 503 normalization to project names and extras: lowercase and collapse runs of ., _, and - to a single -.
- Sorts and deduplicates extras after normalization.
- Removes whitespace around version comparison operators.
- Preserves meaningful version and marker content.
- Drops empty dependency entries when keys are computed.
Examples:
- Django and django normalize identically.
- Flask-Login, flask_login, and flask.login normalize identically.
- requests[Socks,security] becomes requests[security,socks].
- requests < 3 becomes requests<3.
This is intentionally not a complete PEP 508 parser. Inputs it cannot meaningfully parse remain deterministic without attempting semantic normalization.
normalizeInterpreterPath(interpreterPath)

Uses the existing normalizePath helper so Windows path casing and separator differences do not fragment the cache. POSIX path casing remains unchanged.
computeCacheKey({ dependencies, interpreterPath })
- Normalizes, deduplicates, and sorts dependencies.
- Normalizes the interpreter path.
- Builds a versioned, labelled payload.
- Hashes the payload with SHA-256.
- Returns the first 16 lowercase hexadecimal characters.
The resulting key is deterministic and safe to use as a directory name under the cache root introduced by PR 2.
CACHE_KEY_HEX_LENGTH

Exposes the key length so callers and tests do not duplicate the value 16.

Caller contract

interpreterPath must be absolute, trimmed, and already resolved through symlinks, for example with fs.realpath().

The utility deliberately performs no I/O. Two different path strings that resolve to the same executable will therefore produce different keys unless the caller canonicalizes them first. The creation flow in PR 5 will own that resolution.

Cache behavior

The script path and raw requires-python value are intentionally not included directly in the key.

Two scripts with equivalent dependencies and the same selected interpreter share an environment.
Moving or renaming a script does not invalidate its environment.
Adding, removing, or repinning a dependency produces a new key.
Selecting a different interpreter produces a new key.
A later PR re-verifies requires-python on cache hits in case the constraint changes without changing the selected interpreter.

This supports the design's two outcomes: reuse an unchanged cache entry or build a fresh environment. There is no in-place dependency synchronization path.

Tests

Adds src/test/common/inlineScriptCacheKey.unit.test.ts with 37 unit tests covering:

PEP 503 project-name normalization.
Extras normalization, sorting, deduplication, and whitespace handling.
Version comparison operators and multi-clause specifiers.
Empty and duplicate dependency entries.
Dependency ordering and deterministic output.
Inputs that should and should not change the key.
Windows and POSIX interpreter-path behavior.
Fixed-length lowercase hexadecimal output.
Filesystem-safe output.
Pinned caller-contract behavior such as trailing whitespace in interpreter paths.

Platform behavior is stubbed through platformUtils, keeping the suite independent of the host operating system.

User impact

Zero.

Nothing in the extension calls these helpers yet. This PR adds no settings, commands, UI, logging, filesystem writes, or activation behavior. The utility is wired into environment creation in PR 5.

Design context

This implements the cache-key portion of Q4 in #1601: a pipx-style cache keyed by normalized dependencies and the selected interpreter.

PEP 503 normalization for project names and extras was added in response to review feedback on the design PR.

brettcannon · 2026-07-15T21:30:40Z

The normalization approach SGTM.

eleanorjboyd · 2026-07-20T15:27:38Z

+ * deduplicated, and sorted alphabetically — same canonical form pip and
+ * uv use for the project name, applied to each extra.
+ */
+export function normalizeDependency(dep: string): string {


wondering if all of these should live in a PEP 508 file so ai can more easily find and re-use these helpers at other points

Yes it makes sense to move the normalizePEP503Name to a util file for common use, I will do that. But I would keep the remaining dependency canonicalization there because it intentionally implements only cache-equivalent transformations, not full PEP 508 parsing, treating it as reuseable could lead to incorrect use.

Pure utility for computing a deterministic cache key for a PEP 723 inline-script environment, given the script's declared dependencies and the chosen Python interpreter path. First foundation PR for inline-script env support; no behavior impact on its own -- nothing in the extension calls these helpers yet. What is in: - src/common/inlineScriptCacheKey.ts - normalizeDependency(dep) -- folds common variants of the same PEP 723 requirement so trivial edits don't fragment the cache: * PEP 503 name normalization on the project name AND on each extra: lowercase, then collapse runs of [._-] to a single -. So `Django` ≡ `django`, `Flask-Login` ≡ `flask_login`, `requests[Socks]` ≡ `requests[socks]`, `pkg[socks_5]` ≡ `pkg[socks-5]`. * Extras are deduplicated and sorted alphabetically after canonicalization, so `requests[security,socks]` and `requests[socks,security]` hash to the same key. * Whitespace around comparison operators is stripped, so `requests <3` ≡ `requests<3`. Not a full PEP 508 parser -- only folds the superficial edits users are likely to make. Pinned-behavior tests document what is and isn't folded. - normalizeInterpreterPath(path) -- wraps the existing normalizePath helper so case/separator differences on Windows do not fragment the cache. POSIX paths pass through unchanged. - computeCacheKey({ dependencies, interpreterPath }) -- normalizes and dedupes deps, sorts, builds a versioned labelled-line payload, hashes with SHA-256, and truncates to 16 hex characters (64 bits -- well below practical collision risk for a per-user cache). - src/test/common/inlineScriptCacheKey.unit.test.ts -- 37 unit tests covering the three exports, including pinned-behavior tests for asymmetries the docstrings call out (trailing whitespace in paths fragments) and explicit canonicalization tests for extras (PEP 503 folding, sort, dedup, whitespace tolerance, empty-bracket collapse, composition with version specifiers). Caller contract The interpreter path must be absolute and symlink-resolved before calling computeCacheKey. The function does no I/O, so symlink canonicalization is the caller's responsibility -- a later PR will do that at the resolution site. Design context Implements the cache-key half of PR 1 in the Phase 1 roadmap of pep723_design_questions.md. Deps-keyed + interpreter-path-keyed matches the pipx-style choice in Q4. The PEP 503 extras handling was added in response to reviewer feedback on the design doc. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…guard (PEP 723 PR 2/16) (#1635) > Part of #1602 (PEP 723 inline script env support). Design doc: #1601. ### Roadmap context This is **PR 2 of 16** in the PEP 723 inline-script roadmap. The full plan lives in #1602. | Phase | PR | Status | |---|---|---| | **Phase 1: Foundation** | PR 1: cache key hash utility | in review (#1634) | | | **PR 2: cache layout + `meta.json` sidecar** | **this PR** | | | PR 3: `requires-python` to interpreter selection | _in progress_ | | **Phase 2: Manager** | PR 4: `InlineScriptEnvManager` skeleton | merged (#1610) | | | PR 5: `create()` happy path | not started (needs 1, 2, 3, 4) | | | PR 6: `create()` uv-install fallback | not started (needs 3, 5) | | | PR 7: persistence with `get`, `set`, and Memento | _in progress_ (needs 4) | | | PR 8: activation-time discovery | not started (needs 2, 4, 7) | | **Phase 3: Routing** | PR 9: route PEP 723 scripts to the inline manager | not started (needs 4, 7) | | | PR 10: per-script project registration | not started (needs 9) | | **Phase 3.5: Cross-repo** | PRs 17-19: Pylance and vscode-python integration | not started | | **Phase 4: UX** | PR 11: picker item; PR 12: bulk command | not started | | **Phase 5: Lifecycle and polish** | PR 13: clear cache; PR 14: TTL; PR 15: telemetry; PR 16: status bar | not started | PRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path. ### Why this PR Q2 and Q3 of #1601 place inline-script environments under `<globalStorageUri>/script-envs-v1/<cache-key>/` and give each environment a `.meta.json` sidecar for lifecycle bookkeeping. Those contracts need to exist before the manager can create, discover, reuse, or clean up cached environments. This PR also implements the cache-hit usability guard from Q4 step 4. Hashing the selected interpreter path catches a switch to a different Python path, but it does not catch an interpreter being uninstalled from the same path. The guard detects that stale cache state before the extension attempts to reuse a broken environment. The filesystem concerns are kept separate from PR 1's pure hashing logic and PR 3's interpreter-selection logic. ### What this PR does Adds `src/common/inlineScriptCacheLayout.ts` with cache path helpers, typed sidecar I/O, stale-entry selection, and an environment-usability check. #### Cache layout - `getScriptEnvCacheRoot(globalStorageUri)` returns `<globalStorageUri>/script-envs-v1/`. - `getScriptEnvDir(globalStorageUri, cacheKey)` returns the directory for one cached environment. - `getMetaJsonPath(envDir)` returns `<envDir>/.meta.json`. - `INLINE_SCRIPT_CACHE_DIR_NAME`, `META_JSON_FILENAME`, and `META_SCHEMA_VERSION` centralize the on-disk contract. The cache directory and metadata schema are both versioned. An incompatible future format should bump the cache directory suffix and schema version together rather than migrate environments in place. #### Typed `.meta.json` sidecar `InlineScriptEnvMeta` contains the fields with concrete downstream consumers: - `schemaVersion`: validates the sidecar format. - `scriptFsPath`: identifies the owning script for lifecycle cleanup. - `lastUsedAt`: drives TTL eviction. - `requiresPython?`: supports compatibility revalidation on cache hits. `readMetaJson(envDir)` never throws. It returns `undefined` with a warning for missing or non-regular files, files larger than 1 MiB, malformed JSON, unknown schema versions, invalid field types, and non-canonical ISO timestamps. Unknown properties are dropped when the validated object is constructed. `writeMetaJson(envDir, meta)` ensures the directory exists, writes formatted JSON to a randomized sibling temporary file, and atomically renames it to `.meta.json`. If writing or renaming fails, it attempts to remove the temporary file and rethrows the original error. #### TTL selection `selectStaleEntries(entries, now, ttlMs)` is a pure selector used by the future opportunistic cleanup path. - An entry is stale only when its age is strictly greater than the TTL. - Entries exactly at the TTL boundary are retained. - Future timestamps are retained. - Entries without `lastUsedAt` are retained because the extension should not delete data it cannot classify confidently. The function returns paths to delete; it performs no filesystem walk or deletion itself. #### Cache-hit usability guard `verifyEnvUsable(envDir)` checks whether the cached environment still has a usable base interpreter. It never throws; failures produce a warning and return `false`. - **POSIX:** stats `<envDir>/bin/python`. Because `fs.stat` follows symlinks, a dead launcher symlink is detected. - **Windows:** reads `home = ...` from `pyvenv.cfg` and stats `<home>/python.exe`. Checking `<envDir>/Scripts/python.exe` would be insufficient because that launcher can remain after the base Python is uninstalled. This covers common removal paths such as `pyenv uninstall`, `uv python uninstall`, Homebrew or apt removal, and Windows Add/Remove Programs. ### Failure policy The helpers distinguish recoverable cache state from creation-time failures: - Metadata reads and usability checks return `undefined` or `false` rather than throwing, allowing callers to treat invalid cache state as a miss. - Metadata writes rethrow because a caller must know that newly created state was not recorded successfully. - The TTL selector is pure and cannot delete anything by itself. ### Tests Adds `src/test/common/inlineScriptCacheLayout.unit.test.ts` with 44 focused unit tests covering: - Cross-platform cache and sidecar path construction. - Metadata write/read round trips and concurrent writers. - Atomic write behavior and temporary-file cleanup. - Missing, malformed, oversized, and structurally invalid sidecars. - Schema-version and canonical-timestamp validation. - TTL boundaries, future timestamps, and missing `lastUsedAt` values. - POSIX launchers, live symlinks, dead symlinks, and non-file paths. - Windows `pyvenv.cfg` parsing and missing base interpreters. Required checks pass on this branch: - `npm run lint` - `npm run compile-tests` - `npm run unittest`: 1418 passing, 0 failing, 4 pending ### User impact **Zero.** Nothing in the extension calls these helpers yet. This PR creates no environments, writes nothing to global storage during activation, deletes no cache entries, and adds no settings, commands, UI, or logging on the default execution path. PR 5 will use the layout and sidecar helpers during creation. PRs 7 and 8 will use them during reuse and activation-time discovery. PRs 13 and 14 will use the cleanup helpers. ### Design context This implements Q2 (disk location), Q3 (environment contents and sidecar metadata), Q4 step 4 (base-interpreter existence guard), and the pure selection portion of Q7 (TTL cleanup) from #1601. The base-interpreter guard was added in response to review feedback on the design PR. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

) > Part of #1602 (PEP 723 inline script env support). Design doc: #1601. ### Roadmap context This is **PR 3 of 16** in the PEP 723 inline-script roadmap. The full plan lives in #1602. | Phase | PR | Status | |---|---|---| | **Phase 1: Foundation** | PR 1: cache key hash utility | in review (#1634) | | | PR 2: cache layout + `meta.json` sidecar | in review (#1635) | | | **PR 3: `requires-python` to interpreter selection** | **this PR** | | **Phase 2: Manager** | PR 4: `InlineScriptEnvManager` skeleton | merged (#1610) | | | PR 5: `create()` happy path | not started (needs 1, 2, 3, 4) | | | PR 6: `create()` uv-install fallback | not started (needs 3, 5) | | | PR 7: persistence with `get`, `set`, and Memento | _in progress_ (needs 4) | | | PR 8: activation-time discovery | not started (needs 2, 4, 7) | | **Phase 3: Routing** | PR 9: route PEP 723 scripts to the inline manager | not started (needs 4, 7) | | | PR 10: per-script project registration | not started (needs 9) | | **Phase 3.5: Cross-repo** | PRs 17-19: Pylance and vscode-python integration | not started | | **Phase 4: UX** | PR 11: picker item; PR 12: bulk command | not started | | **Phase 5: Lifecycle and polish** | PR 13: clear cache; PR 14: TTL; PR 15: telemetry; PR 16: status bar | not started | PRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path. ### Why this PR Before the inline-script manager can create an environment, it needs to answer two questions from Q4 of #1601: 1. Which installed base interpreter is the newest one compatible with the script's `requires-python` declaration? 2. If none is compatible, what lower-bound version can the existing uv installation flow request? Both decisions are pure functions over an environment list and a PEP 440 specifier. Keeping them outside the manager makes the filtering, ranking, and fallback rules independently testable without environment discovery, UI, filesystem access, or process execution. ### What this PR does Adds `src/common/inlineScriptInterpreter.ts` with two exported helpers. #### `pickCompatibleInterpreter(installed, requiresPython)` Filters the supplied environments and returns the newest usable Python 3 interpreter satisfying the script's constraint. A candidate is rejected when: - `env.error` is set. - `env.version` is missing, empty, or does not begin with numeric release segments. - The interpreter is not Python 3. - The version does not satisfy `requiresPython` according to the existing `matchesPythonVersion` helper. An undefined, empty, or whitespace-only constraint is treated as unconstrained. If no candidate qualifies, the function returns `undefined` so the creation flow can offer the uv fallback. Candidates are ranked by numeric release segments in descending order. A leading `v` is tolerated, and prerelease, development, and local suffixes are ignored for ranking. Ties preserve input order. The input array is copied before sorting and is never mutated. Examples: - No constraint across Python 3.10, 3.11, and 3.12 selects 3.12. - `>=3.11,<3.13` selects the newest interpreter within that range. - `==3.12.*` uses the existing wildcard-matching semantics. - `~=3.12.4` honors the compatible-release upper bound. - If only Python 2, broken environments, or incompatible versions are available, the result is `undefined`. #### Base-interpreter caller contract `installed` must contain base interpreters only: system Python, pyenv-installed Python, uv-installed Python, or conda `base`. Derived environments such as venvs, named conda environments, Poetry environments, and Pipenv environments must not be used as venv bases. `api.getEnvironments('global')` is the intended source. The helper still defensively rejects errored, versionless, unparseable, and non-Python-3 entries. #### `extractLowerBoundVersion(requiresPython)` Extracts the tightest usable lower bound for a future `uv python install <version>` request. Supported floor-producing forms include: - `>=3.13` to `3.13` - `~=3.12.4` to `3.12.4` - `==3.12.7` to `3.12.7` - `==3.12.*` to `3.12` - `>=3.11,>=3.12,<3.14` to `3.12` When multiple lower bounds are present, the numerically greatest one is returned. Recognized operators that do not provide a safe integer floor (`>`, `<`, `<=`, `!=`, and `===`) are skipped without warning. Malformed clauses, invalid wildcard placement, and `~=` values without at least major and minor segments are skipped with `traceWarn`. A skipped clause does not discard a valid floor from another clause. For example, `>=3.11,<3.13` returns `3.11`, while an upper-bound-only spec returns `undefined`. The extracted value is an installation hint, not a full compatibility result. PR 6 will re-run `matchesPythonVersion` after uv installs an interpreter. ### Tests Adds `src/test/common/inlineScriptInterpreter.unit.test.ts` with 34 focused unit tests covering: - Empty input and no-compatible-interpreter results. - Unconstrained and constrained newest-version selection. - Multi-clause, wildcard, and compatible-release constraints. - Errored, versionless, unparseable, and Python 2 environments. - Empty-string constraints and leading `v` prefixes. - Stable ties and non-mutating input behavior. - Ranking versions with prerelease, development, and local suffixes. - Lower-bound extraction for `>=`, `==`, wildcard `==`, and `~=`. - Tightest-floor selection across multiple clauses. - Upper-bound-only and unsupported operators. - Malformed clauses, wildcard misuse, and warning behavior. Required checks pass on this branch: - `npm run lint` - `npm run compile-tests` - `npm run unittest`: 1418 passing, 0 failing, 4 pending ### User impact **Zero.** Nothing in the extension calls these helpers yet. This PR discovers no interpreters, installs no Python versions, creates no environments, and adds no settings, commands, UI, filesystem access, or activation behavior. PR 5 will call `pickCompatibleInterpreter` for the normal creation path. PR 6 will call `extractLowerBoundVersion` when no compatible installed interpreter exists and the user is offered a uv installation. ### Design context This implements Q4 step 1 from #1601: select the newest installed Python satisfying `requires-python`, and derive a lower-bound version for the uv fallback when no installed interpreter qualifies. It reuses the existing `matchesPythonVersion` implementation rather than introducing a second PEP 440 matcher. ### Explicit installation consent Following review feedback, the uv fallback now has an explicit, informed-consent contract for inline scripts: - The modal explains the script's `requires-python` constraint and the exact Python version requested. - If uv is missing, both the message and action disclose that uv and Python will be installed. - No prompt is offered when a compatible concrete install version cannot be derived. - Script-controlled prompt text is flattened, bounded, and stripped of control characters; install versions must be numeric release segments. - Dismissal performs no installation and remains retryable; inline-script setup does not inherit the global "Don't ask again" suppression state. - Approval forwards the displayed version to `uv python install`, and installed-version lookup matches release-segment boundaries. The inline-script manager still has no creation call site in this PR, so this establishes and tests the consent API without changing current user behavior. PR 6 will invoke it from the uv fallback. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

StellaHuang95 added the feature-request Request for new features or functionality label Jul 14, 2026

StellaHuang95 requested a review from eleanorjboyd July 14, 2026 21:04

This was referenced Jul 15, 2026

Add inline-script cache layout, meta.json sidecar, and env-usability guard (PEP 723 PR 2/16) #1635

Merged

Add inline-script interpreter selection helpers (PEP 723 PR 3/16) #1636

Merged

StellaHuang95 requested a review from edvilme July 17, 2026 22:05

eleanorjboyd previously approved these changes Jul 20, 2026

View reviewed changes

StellaHuang95 force-pushed the pep723-pr1-cache-hash branch from 104b22d to c84ba89 Compare July 20, 2026 21:17

refactor(inline-scripts): reuse package name normalization

6742524

StellaHuang95 dismissed eleanorjboyd’s stale review via 6742524 July 20, 2026 22:13

eleanorjboyd approved these changes Jul 21, 2026

View reviewed changes

rzhao271 approved these changes Jul 21, 2026

View reviewed changes

StellaHuang95 merged commit e7ea93c into microsoft:main Jul 21, 2026
45 checks passed

StellaHuang95 deleted the pep723-pr1-cache-hash branch July 21, 2026 16:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add inline-script cache key hash utility (PEP 723 PR 1/16)#1634

Add inline-script cache key hash utility (PEP 723 PR 1/16)#1634
StellaHuang95 merged 2 commits into
microsoft:mainfrom
StellaHuang95:pep723-pr1-cache-hash

StellaHuang95 commented Jul 14, 2026

Uh oh!

brettcannon commented Jul 15, 2026

Uh oh!

eleanorjboyd Jul 20, 2026

Uh oh!

StellaHuang95 Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

StellaHuang95 commented Jul 14, 2026

Roadmap context

Why this PR

What this PR does

Caller contract

Cache behavior

Tests

User impact

Design context

Uh oh!

brettcannon commented Jul 15, 2026

Uh oh!

eleanorjboyd Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

StellaHuang95 Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants