Skip to content

feat: set BentoML user agent on the S3 client used by import/export - #5680

Draft
goanpeca wants to merge 2 commits into
bentoml:mainfrom
goanpeca:feat/s3-user-agent
Draft

feat: set BentoML user agent on the S3 client used by import/export#5680
goanpeca wants to merge 2 commits into
bentoml:mainfrom
goanpeca:feat/s3-user-agent

Conversation

@goanpeca

Copy link
Copy Markdown

What does this PR address?

bentoml models export/import (and the Bento equivalents) build their fsspec filesystem without passing any client configuration, so requests to s3:// targets go out with the default botocore user agent. Nothing in an access log or a support ticket identifies the traffic as coming from BentoML.

This PR appends BentoML/<version> to the user agent of the S3 client used by import/export, and documents how to point the Model Store at a non-AWS, S3-compatible endpoint.

Changes

src/bentoml/_internal/exportable.py

  • Add _s3_storage_options(protocol), which returns {"s3": {"config_kwargs": {"user_agent_extra": f"BentoML/{BENTOML_VERSION}"}}} when the protocol is s3, and {} otherwise. BENTOML_VERSION is imported inside the helper so the module import stays unchanged.
  • Thread those options through the three places import/export constructs a filesystem: fsspec.url_to_fs() on the import path, fsspec.url_to_fs() on the export path, and target_options= for ZipFileSystem / TarFileSystem, so archives read straight out of S3 are configured too.

docs/source/build-with-bentoml/model-loading-and-management.rst

  • Note that the s3:// scheme works with Amazon S3 and with any S3-compatible object store (Backblaze B2, Cloudflare R2, MinIO), and that the standard AWS_ENDPOINT_URL environment variable selects a non-AWS endpoint. Includes a two-line example.

Notes

  • Additive only. user_agent_extra is botocore's documented hook for appending to the user agent string, so credentials, endpoint_url, region and any other storage option a user already passes are left untouched. Non-s3 protocols (file, gs, abfs, ...) receive an empty dict and behave exactly as before.
  • {"s3": {...}} is fsspec's protocol-keyed storage-option form, which both url_to_fs() and the chained target_options understand, so the same helper works for the plain and archive code paths.
  • No new dependency, and no change to request semantics or payloads.
  • No tests added: the change is a keyword-argument pass-through, and asserting on it end to end would mean asserting the shape of s3fs/botocore internals. Happy to add a unit test over _s3_storage_options() if you would prefer one.

Before submitting:

  • Does the Pull Request follow Conventional Commits specification naming?
  • Does the code follow BentoML's code style? ruff-check and ruff-format are clean on the changed file.
  • Did you read through contribution guidelines and follow development guidelines?
  • Did your changes require updates to the documentation? Have you updated those accordingly?
  • Did you write tests to cover your changes? (see Notes)

goanpeca added 2 commits July 30, 2026 08:39
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant