Skip to content

Commit 165d400

Browse files
Add inline script activation-time discovery (PEP 723 PR 8/16) (#1722)
> Part of #1602 (PEP 723 inline script env support). Design doc: #1601. > > Builds on the merged persistence work in #1697 and is rebased on current `main`. ### Roadmap context This is **PR 8 of 16** in the PEP 723 inline-script roadmap. It makes extension-owned cached environments discoverable after activation; automatic script routing remains a separate follow-up. | Phase 2: Manager | PR | Status | |---|---|---| | | PR 4: `InlineScriptEnvManager` skeleton | merged (#1610) | | | PR 5a: generic env-creation utilities | merged (#1651) | | | PR 5b: inline-script cache + interpreter utilities | merged (#1655) | | | PR 5c: `create()` happy path | merged (#1656) | | | PR 6: `create()` uv-install fallback | merged (#1696) | | | PR 7: persistence (`get` / `set` + Memento) | merged (#1697) | | | **PR 8: activation-time discovery** | **this PR** | | | PR 9: route PEP 723 scripts to the inline manager | follow-up | ### Why this PR PR 7 persists and lazily rehydrates a specific script's selected inline environment when the manager is asked for that script. The manager still has no global inventory, however: `getEnvironments()` returns `[]`, and cached environments are not published after an extension-host restart unless a script-specific lookup happens to reconstruct one. This PR implements the activation-discovery portion of Q4 in the design: - walk the versioned global cache without blocking extension activation; - validate each cache entry before exposing it as a `PythonEnvironment`; - publish add/remove events as valid entries appear or become definitively invalid; - preserve previously known entries through locks and transient filesystem failures; - retry bounded transient work without creating a permanent watcher or polling loop. ### What this PR does **Defers discovery until after manager registration** - Starts activation discovery with `setImmediate()` after the manager is registered. - Leaves the feature-gate-off path unchanged: no manager registration, cache scan, timer, or filesystem work. - Keeps extension activation non-blocking. **Publishes a global discovered collection** - `getEnvironments('all')` returns a copy of the validated cache collection. - Other scopes remain empty because inline cache entries live in extension global storage rather than belonging to one workspace. - Results are sorted deterministically. - Collection reconciliation emits precise `EnvironmentChangeKind.add` and `remove` events only when manager identity, executable path, or Python version materially changes. **Validates every cache entry before publication** A candidate must: 1. be a normal directory rather than a symlink; 2. be unlocked and physically contained beneath the expected cache root; 3. have a valid `.meta.json` sidecar; 4. have an available cached launcher and base interpreter; 5. resolve into a real `PythonEnvironment`; 6. prove direct-child cache ownership through `sysPrefix`; and 7. match the Python release recorded in the sidecar. Discovery is read-only. It does not delete, rebuild, or rewrite invalid entries. **Distinguishes definitive invalidity from uncertainty** - Missing, malformed, unowned, or version-mismatched entries are omitted. - Locked entries and transient I/O/resolver/ownership failures preserve the previously published environment and request another activation pass. - A missing cache root is treated as an empty cache. - An unavailable cache root preserves the current collection and remains retryable. **Handles concurrent cache changes** - Activation scans compare an initial and final directory snapshot so an entry created during the scan triggers a follow-up pass. - Final cache-root absence publishes an empty collection immediately; transient final-read failures preserve the prior collection until a retry. - Per-entry filesystem fingerprints detect a cache directory rebuilt under the same key even when its name is unchanged. - Locks use the cache entry name/hash as their identity. - Published entries use that same cache-key identity rather than mixing lexical cache paths with canonical `sysPrefix` paths, avoiding false removal through symlink/junction path differences. - Lock probing uses `lstat`; only `ENOENT` means unlocked. `EIO`, access failures, and other uncertain states fail closed. **Coalesces refresh work without weakening activation discovery** - Concurrent compatible refreshes share one scan. - Activation joining an in-flight explicit refresh receives one snapshot-aware follow-up rather than accepting the explicit pass's weaker single-snapshot semantics. - Explicit `refresh()` cancels activation retries, performs one settled pass, and schedules no delayed work afterward. **Retries activation discovery only while useful** - Follow-up delays are bounded at 1 second, 5 seconds, and 30 seconds. - Retries are requested for locks, transient failures, or a changed cache snapshot. - No permanent filesystem watcher or unbounded polling loop is introduced. - Disposal cancels pending timers and prevents in-flight scans from publishing late results. **Strengthens the Windows usability guard** - Windows cache validation now checks both the cached environment launcher and the base interpreter referenced by `pyvenv.cfg`. - A surviving base interpreter no longer makes an environment with a missing `Scripts\python.exe` appear usable. ### Discovery semantics | Cache state | Behavior | |---|---| | Valid sidecar, launcher, interpreter, ownership, and version | Publish environment | | Entry is locked or being built | Preserve previous publication; retry | | Filesystem/resolver state is temporarily unavailable | Preserve previous publication; retry | | Entry appears during the scan | Schedule snapshot-aware follow-up | | Missing or malformed sidecar | Omit/remove from collection | | Missing launcher/base interpreter | Omit/remove from collection | | Ownership or recorded-version mismatch | Omit/remove from collection | | Cache root is absent | Publish empty collection | ### Example ```text extension activation → register InlineScriptEnvManager → defer one event-loop turn → scan <globalStorage>/script-envs-v1 → validate sidecar + launcher + ownership + version → publish valid cached environments through getEnvironments('all') → retry only if the scan observed a lock, transient state, or snapshot change ``` This inventory does not associate an environment with a script. PR 7 owns persisted script associations, and PR 9 will use those associations for automatic per-file routing. ### Tests Coverage includes: - deferred feature-gated activation startup; - valid cache discovery and `all`-scope publication; - missing, malformed, unavailable, non-directory, and symlinked entries; - missing and non-regular Windows launchers; - lock preservation, including unavailable (`EIO`) lock probes; - canonical `sysPrefix` versus lexical cache-root identities; - add/remove event reconciliation; - concurrent refresh coalescing; - activation joining an explicit refresh; - cache entries created during a scan; - builds completing after the short retry window; - bounded retry exhaustion; - explicit single-pass refresh behavior; and - disposal during in-flight scans and pending retries. Validation on the rebased branch: - `npm run compile-tests` - `npm run compile` - `npm run lint` - focused activation-discovery/cache-launcher/registration suites The full Windows unit run reaches 1613 passing and 5 pending; the existing concurrent `writeMetaJson` rename test can still intermittently fail with `EPERM` on Windows. That writer is unchanged by this PR and the same failure is reproducible on `main`. ### Performance - Activation is deferred and never waits for discovery. - Cache scans are coalesced. - Retries are bounded and stop after a stable pass. - Explicit refresh remains single-pass. - There is no persistent watcher, unbounded polling, or per-document work. ### User impact **No default-path user impact.** The manager and discovery remain behind the undeclared, default-off `python-envs.inlineScripts.enabled` flag. With the internal flag manually enabled, valid cached inline environments become available through the manager after restart. This PR does not automatically select one for a script and introduces no public command, setting, picker item, project registration, cache deletion, TTL cleanup, or telemetry. ### Scope and follow-up This PR intentionally does **not** implement: - automatic PEP 723 script routing (PR 9); - exact script project registration (PR 10); - CodeLens or bulk setup UX (PRs 11-12); - cache clearing or TTL eviction (PRs 13-14); or - lifecycle telemetry (PR 15). PR 7 (#1697) is merged, and this branch is rebased on current `main`. Automatic routing can follow independently after this manager-discovery layer lands. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3cb82ae9-7424-40a4-9156-8c54ac6e0895 Copilot-Session: 6b12d843-8011-4bfc-9ba9-f75761eadee2
1 parent fa47921 commit 165d400

6 files changed

Lines changed: 1169 additions & 8 deletions

File tree

src/common/inlineScript/cacheLayout.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -204,7 +204,7 @@ export function selectStaleEntries(entries: ReadonlyArray<CacheEntrySummary>, no
204204
}
205205

206206
/**
207-
* Verify that a cached env's base interpreter still exists on disk.
207+
* Verify that a cached env's launcher and base interpreter still exist on disk.
208208
*/
209209
export async function verifyBaseInterpreterExists(envDir: Uri): Promise<boolean> {
210210
return (await getBaseInterpreterStatus(envDir)) === 'available';
@@ -221,6 +221,11 @@ async function getPosixBaseInterpreterStatus(envDir: Uri): Promise<BaseInterpret
221221
}
222222

223223
async function getWindowsBaseInterpreterStatus(envDir: Uri): Promise<BaseInterpreterStatus> {
224+
const launcherStatus = await getRegularFileStatus(getVenvPythonPath(envDir.fsPath), 'cached interpreter launcher');
225+
if (launcherStatus !== 'available') {
226+
return launcherStatus;
227+
}
228+
224229
const pyvenvPath = Uri.joinPath(envDir, 'pyvenv.cfg').fsPath;
225230
let raw: string;
226231
try {

0 commit comments

Comments
 (0)