Skip to content

feat(files): fsspec backend so any filesystem can hold uploaded files - #977

Closed
daavoo wants to merge 1 commit into
feat/files-api-parityfrom
feat/files-fsspec
Closed

daavoo wants to merge 1 commit into
feat/files-api-parityfrom
feat/files-fsspec

Conversation

@daavoo

@daavoo daavoo commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #976 (feat/files-api-parity); only the last commit is this PR.

What

files_backend: fsspec with files_url (gcs://bucket/prefix, abfs://container/prefix, s3://bucket/prefix, sftp://host/path, file:///path, memory://) and files_storage_options for the implementation's own keyword arguments (credentials, endpoints, projects). FsspecFileStore implements the same FileStore protocol as the local and boto3 S3 backends: writes and reads stream through worker threads, a partial upload is removed on failure or cancellation, backend exceptions are translated to the OSError family callers already handle, and a ref that could leave the root is refused.

fsspec was already in the dependency tree through any-llm and is now declared; the lock gains only that entry. files_storage_options is kept out of the settings view as a credential. Operators install the implementation package for their protocol (gcsfs, adlfs, s3fs, paramiko).

Validation

Unit tests run on fsspec's built-in memory:// and file:// filesystems, so no cloud package or network is needed. make lint, make typecheck, unit suite, OSS smoke gate, and the files/config/settings integration tests pass.

🤖 Generated with Claude Code

files_backend: fsspec plus a files_url (gcs://, abfs://, s3://, sftp://,
file://, ...) and files_storage_options reach whatever filesystem fsspec has an
implementation installed for, through the same FileStore protocol the local
and boto3 S3 backends implement. fsspec was already in the tree through
any-llm and is now a declared dependency.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the missing-template PR is missing required template sections label Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ PR Template Missing

This PR appears to be missing required sections from the PR template.

Please edit your PR description to include the PR Type, Checklist, and AI Usage sections. The template helps maintainers review your contribution.

This PR will be automatically closed in 24 hours if the template is not restored.

If you're using an AI coding tool, please ensure it preserves the PR template.

@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically closed because the required template sections were not restored within 24 hours.

Please create a new PR using the template and complete the checklist. The template helps maintainers review your contribution.

@daavoo

daavoo commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #1364 together with #976, rebased onto current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-template PR is missing required template sections

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant