Skip to content

feat(environments): mount Memory Stores into session sandboxes - #331

Open
yangyings wants to merge 6 commits into
superduck-ai:mainfrom
yangyings:feat/memory-sandbox-e2e
Open

yangyings wants to merge 6 commits into
superduck-ai:mainfrom
yangyings:feat/memory-sandbox-e2e

Conversation

@yangyings

@yangyings yangyings commented Sep 6, 2026

Copy link
Copy Markdown

Summary

Stack

  1. feat(memory): session/deployment attach contract and console form #329 feat/memory-attach-console
  2. feat(filestore): writable /memory/{slug} namespace writeback #330 feat/memory-filestore
  3. This PR (feat(environments): mount Memory Stores into session sandboxes #331) — 1 commit on top of feat(filestore): writable /memory/{slug} namespace writeback #330

Review order: 1 → 2 → 3.

Still targets main; GitHub Files includes #329 and #330 until those merge. Review only the sandbox commit (feat(environments): mount Memory Stores into session sandboxes), or git diff feat/memory-filestore...feat/memory-sandbox-e2e.

Test plan

  • Session start mounts attached stores and writes MEMORY.md
  • Console store list/tree/Add memory hide /MEMORY.md
  • Agent can read/write via Filestore memory namespace from sandbox
  • just test-e2e-memory-sandbox / workflow when e2b key present
  • Unit tests under internal/environments and tests/environments_memory_* pass

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 2f83eb81-9a98-420e-bd11-05cccacd669b

📥 Commits

Reviewing files that changed from the base of the PR and between 4e40744 and 1d5f660.

📒 Files selected for processing (83)
  • .github/workflows/e2e-memory-sandbox.yml
  • docs/design/be/deployments-api-contract.md
  • docs/design/be/e2b-sandbox-image-contract.md
  • docs/design/be/filestore.md
  • docs/design/be/memory-store.md
  • docs/design/be/messages-proxy.md
  • docs/design/fe/memory-store-attach-form.md
  • docs/design/fe/session-file-resources-form.md
  • docs/design/fe/sessions/session-conversation-workspace.md
  • internal/db/memory.go
  • internal/db/memory_mapper.go
  • internal/db/memory_mapper.xml
  • internal/db/memory_mapper_test.go
  • internal/db/session_file_mounts_test.go
  • internal/db/session_memory_resources.go
  • internal/db/session_resource_mapper.go
  • internal/db/session_resource_mapper.xml
  • internal/db/session_resource_mapper_test.go
  • internal/db/sessions.go
  • internal/db/sessions_helpers.go
  • internal/deployments/execution.go
  • internal/deployments/execution_test.go
  • internal/deployments/handler.go
  • internal/deployments/handler_contract_test.go
  • internal/deployments/resources.go
  • internal/deployments/worker.go
  • internal/environments/environment_manager.go
  • internal/environments/environment_manager_test.go
  • internal/environments/managed_agent_runtime_resources.go
  • internal/environments/memory_runtime.go
  • internal/environments/memory_runtime_test.go
  • internal/environments/rclone_filestore.go
  • internal/environments/rclone_filestore_test.go
  • internal/environments/runner.go
  • internal/filestore/memory_backend.go
  • internal/filestore/memory_backend_list.go
  • internal/filestore/memory_backend_list_test.go
  • internal/filestore/memory_backend_move_test.go
  • internal/filestore/memory_path.go
  • internal/filestore/memory_path_test.go
  • internal/filestore/path_backend.go
  • internal/filestore/path_backend_test.go
  • internal/filestore/service.go
  • internal/memory/handler.go
  • internal/sessioncontract/resource.go
  • internal/sessionresource/memory.go
  • internal/sessionresource/memory_test.go
  • internal/sessions/errors.go
  • internal/sessions/errors_test.go
  • internal/sessions/event_payload.go
  • internal/sessions/file_resources_test.go
  • internal/sessions/memory_resource.go
  • internal/sessions/service.go
  • internal/sessions/service_helpers.go
  • internal/vaults/env_credential.go
  • internal/vaults/env_credential_test.go
  • justfile
  • tests/deployments_api_test.go
  • tests/deployments_memory_attach_test.go
  • tests/environments_memory_runtime_test.go
  • tests/environments_memory_sandbox_e2e_test.go
  • tests/environments_runner_cloud_test.go
  • tests/filestore_memory_namespace_test.go
  • tests/sessions_api_test.go
  • tests/sessions_memory_attach_test.go
  • web/src/features/managed-agents/ManagedAgentsPage.resources.suite.tsx
  • web/src/features/managed-agents/ManagedAgentsPage.test-utils.tsx
  • web/src/features/managed-agents/api.test.ts
  • web/src/features/managed-agents/api.ts
  • web/src/features/managed-agents/resources/MemoryStoresAttachField.tsx
  • web/src/features/managed-agents/resources/dialogs.tsx
  • web/src/features/managed-agents/resources/entity-dialog-ready.ts
  • web/src/features/managed-agents/resources/memory-attach.test.ts
  • web/src/features/managed-agents/resources/memory-attach.ts
  • web/src/features/managed-agents/resources/model.test.ts
  • web/src/features/managed-agents/resources/model.tsx
  • web/src/features/managed-agents/sessions/SessionFileResourcesField.tsx
  • web/src/features/managed-agents/sessions/SessionInspector.tsx
  • web/src/features/managed-agents/sessions/session-resource-display.test.ts
  • web/src/features/managed-agents/sessions/session-resource-display.ts
  • web/src/features/managed-agents/types.ts
  • web/src/shared/i18n/messages/en.json
  • web/src/shared/i18n/messages/zh-CN.json

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pullfrog

pullfrog Bot commented Sep 6, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@yangyings
yangyings force-pushed the feat/memory-sandbox-e2e branch from 45c6e06 to c165006 Compare September 7, 2026 03:15
@pullfrog

pullfrog Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@pullfrog

pullfrog Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@pullfrog

pullfrog Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@pullfrog

pullfrog Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@yangyings

Copy link
Copy Markdown
Author

@pullfrog review this pull request

Allow Memory Store attach on session and deployment create paths, persist
mount snapshots, and expose the console Resources UI for attach + display.
Sandbox mount and filestore writeback remain follow-up PRs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@yangyings
yangyings force-pushed the feat/memory-sandbox-e2e branch from 1785998 to d2f009b Compare September 7, 2026 05:41
@pullfrog

pullfrog Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run failed. View the logs →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using openai-compatible/deepseek-v4-flash𝕏

@jh0904

jh0904 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2f009b39c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/e2e-memory-sandbox.yml
Comment thread internal/filestore/service.go
Comment thread internal/filestore/memory_backend.go Outdated
Comment thread internal/filestore/memory_backend.go
Comment thread internal/filestore/memory_backend.go
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-07T06:38:45.189552Z d2f009b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

yangyingshun and others added 2 commits September 8, 2026 11:24
Create Session/Deployment used the list helper's first page of 5, so
stores beyond that could not be selected. Follow next_page at the API limit.
Route Filestore /memory/{slug} through a memory backend so attached stores
write back to Memory tables and object storage.

Co-authored-by: Cursor <cursoragent@cursor.com>
@yangyings
yangyings force-pushed the feat/memory-sandbox-e2e branch from d2f009b to a95e95d Compare September 8, 2026 03:26

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed changes — reviewed the isolated #331 commit (a95e95de), i.e. git diff 0c7346eb..a95e95de; the remaining files in the branch diff belong to the stacked #329/#330 PRs and are out of scope here.

  • Converted memory_store session resources from opaque sources into parsed memoryMounts, failing closed (invalidMemoryResources) on any unparseable snapshot so an attached store is never silently unmounted.
  • Appended one rclone mount per attached store (/memory/{slug}mount_path, 1s cache, token by access), keeping the /mnt/memory parent as local disk rather than a filestore mount.
  • Added startManagedAgentSessionFilesystem (mkdir → rclone ready → write MEMORY.md), fail-closed on mkdir/write errors, and gated to the fresh-sandbox create path so pause/resume does not re-run it.
  • Injected CLAUDE_CODE_REMOTE_MEMORY_DIR / CLAUDE_COWORK_MEMORY_PATH_OVERRIDE into the sandbox env only when stores are mounted, while keeping appendSystemPrompt static (no memory-policy leak); reserved both names in the vault credential layer.
  • Hid the platform /MEMORY.md from the console store list/tree and blocked Add-memory for it.
  • Added the e2e-memory-sandbox workflow/just target plus extensive unit tests and a real-E2B cross-session lifetime suite (M5-*).

✅ No new issues found.

ℹ️ Confirm memory consumption against the pinned claude build

This is a verification ask, not a code defect — the sandbox-side plumbing (mounts, MEMORY.md, env injection) is correct and well-tested. But no test in this commit exercises an actual agent turn persisting memory through /mnt/memory: the E2E writes sandbox files via shell (writeSandboxFile) and asserts MEMORY.md is present and appendSystemPrompt stays clean — it never proves Claude consumes the memory dir.

Both injected env vars are real but undocumented, feature-gated Claude Code knobs (verified names). Claude Code's loadMemoryPrompt ingests a root MEMORY.md verbatim and non-destructively, so the <!-- oma-stores --> store list is at least injected into the model context. However, this PR deliberately inverts the documented Anthropic memory contract (read-only auto-mount + system-prompt store description → local-disk root + env-var memdir) to keep the prompt cache stable, and whether the sandbox's pinned claude build honors these vars under current storage-path semantics (dir as root vs. base) is unverified — if the build implements newer semantics, the env injection could be a silent no-op.

Technical details
# Confirm the env-var memory wiring works on the deployed claude image

## Affected sites
- internal/environments/memory_runtime.go:131-139 — memorySessionEnvironment injects the two vars = /mnt/memory
- internal/environments/runner.go:695-711 — startManagedAgentSessionFilesystem writes /mnt/memory/MEMORY.md
- internal/environments/memory_runtime.go:59-74 — MEMORY.md renders <!-- oma-stores --> + per-store bullets
- tests/environments_memory_sandbox_e2e_test.go — writes via shell, no agent-turn persistence assertion

## Required outcome
- The author confirms the E2B image's pinned claude build honors CLAUDE_CODE_REMOTE_MEMORY_DIR /
  CLAUDE_COWORK_MEMORY_PATH_OVERRIDE as a memdir root containing /mnt/memory/MEMORY.md, OR
- add an E2E step that drives one model turn (e.g. "remember X" / read a store) and asserts the
  memory write lands in the rw store as a session_actor version — proving the model consumes /mnt/memory.

## Open questions for the human
- Which claude version/feature gates run in the E2B template, and which storage-path semantics does it use?
- Is the native (frontmatter topic-file) auto-memory contract intentionally not used here?

Pullfrog  | View workflow run | Using DeepSeek Flash𝕏

yangyingshun and others added 2 commits September 8, 2026 12:40
…ory TTL

Overwrite move deleted the destination before rename committed; identical
createFile left unreferenced objects; ttlSeconds on /memory was accepted
and silently ignored.

Co-authored-by: Cursor <cursoragent@cursor.com>
Provision rclone multimounts, MEMORY.md, and reserved env vars for attached
Memory Stores, and add real E2B lifetime e2e coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>
@yangyings
yangyings force-pushed the feat/memory-sandbox-e2e branch from a95e95d to 09190c3 Compare September 8, 2026 05:00

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the only new commit since the prior Pullfrog review at a95e95de is a8bd1aea; the sandbox commit (09190c3a) is a byte-identical rebase of the already-reviewed a95e95de. Reviewed a8bd1aea fix(filestore):

  • Made overwrite moveFile atomic by extracting applyMemoryUpdate/applyMemoryDelete and adding a single-transaction MoveMemory that soft-deletes the destination and renames the source together, fixing the prior two-transaction delete-then-rename that could lose the destination on a rename failure.
  • Discarded the freshly-uploaded object on a no-op (identical) rewrite via !result.VersionCreated, so identical createFile no longer leaks unreferenced S3 objects.
  • Rejected nonzero ttlSeconds on the memory createFile path (Memory has no expiry) instead of silently ignoring it; the internal rclone writeback path sends no TTL, so there's no regression.
  • Added real-Postgres integration tests and a unit test covering the atomicity/no-op/overwrite-move behavior.

I traced the concurrency and indexing implications and they hold: every memory mutation path (create/update/delete/move) takes the store-row FindForUpdate first, so per-store writers serialize with consistent lock ordering, and the partial unique index on active (memory_store_uuid, path) is what MoveMemory correctly relies on.

Pullfrog  | View workflow run | Using DeepSeek Flash𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the only new commit since the prior Pullfrog review at 09190c3a is 1d5f660e fix(filestore): reject non-UTF-8 memory contents before upload:

  • Added a utf8.Valid(content) guard at the top of upsertMemoryContent, the single shared mutation helper that every memory write path (createFile / copyFile / moveFile and the rclone / FUSE writeback) funnels through, rejecting non-UTF-8 bytes with invalid_argument before any DB write or object-store upload.
  • Updated the filestore.md design doc with the new validity rule and its rationale (REST content is already UTF-8; the Filestore/FUSE writeback path is the only injector of invalid bytes).
  • Added a real-Postgres test writing {0xff,0xfe,0xfd} that asserts a 400 invalid_argument, no memory row created, and no objects uploaded.

I traced the placement and coverage: the check sits before any side effect, is fail-closed, and centralizes the guard so no write path can bypass it. The test can genuinely fail (it asserts an exact error code and the absence of both DB and object-store artifacts), so it is meaningful coverage. No issues found.

Pullfrog  | View workflow run | Using DeepSeek Flash𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants