Skip to content

feat: add gRPC serving for @bentoml.service() - #5707

Open
skundu42 wants to merge 2 commits into
bentoml:mainfrom
skundu42:feat/new-sdk-grpc-serving
Open

feat: add gRPC serving for @bentoml.service()#5707
skundu42 wants to merge 2 commits into
bentoml:mainfrom
skundu42:feat/new-sdk-grpc-serving

Conversation

@skundu42

Copy link
Copy Markdown

What does this PR address?

bentoml serve already type-forks: legacy bentoml.legacy.Service vs @bentoml.service(). bentoml serve-grpc always called serve_grpc_production, which rejects new-style services (Service type doesn't support gRPC serving).

This adds a parallel gRPC stack for @bentoml.service(), mirroring the HTTP new-SDK path:

  • Type-fork in bentoml serve-grpc and bentoml.serve(..., server_type="grpc")
  • v1 unary Call + ServiceMetadata for new-style APIs
  • Codec for str (text), pydantic/dict/list (JSON), np.ndarray (ndarray), and files
  • Streaming, batch, and task APIs return UNIMPLEMENTED with a clear message
  • serialized_bytes / pickle payloads are rejected
  • Legacy 1.1 services still use serve_grpc_production

Fixes #5607
Related #4634

Before submitting:

  • Does the Pull Request follow Conventional Commits specification naming?
  • Does the code follow BentoML's code style, pre-commit run -a script has passed?
  • Did you read through contribution guidelines and follow development guidelines?
  • Did your changes require updates to the documentation? Have you updated those accordingly?
  • Did you write tests to cover your changes?

@skundu42
skundu42 requested a review from a team as a code owner August 26, 2026 11:47
@skundu42
skundu42 requested review from bojiang and removed request for a team August 26, 2026 11:47

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66293f6c43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/_bentoml_impl/server/serving.py
Comment thread src/_bentoml_impl/grpc/codec.py Outdated
Comment thread src/_bentoml_impl/server/grpc/servicer/v1/__init__.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: [serve] serve-grpc failed: <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

1 participant