Skip to content

rust-generated.md

Latest commit

 

History

History
744 lines (576 loc) · 24.4 KB

rust-generated.md

File metadata and controls

744 lines (576 loc) · 24.4 KB

+++ title = "Rust Generated Code Guide" weight = 782 linkTitle = "Generated Code Guide" description = "Describes the API of message objects that the protocol buffer compiler generates for any given protocol definition." type = "docs" +++

This page describes exactly what Rust code the protocol buffer compiler generates for any given protocol definition.

This document covers how the protocol buffer compiler generates Rust code for proto2, proto3, and protobuf editions. Any differences between proto2, proto3, and editions generated code are highlighted. You should read the proto2 language guide, proto3 language guide, or editions guide before reading this document.

Protobuf Rust {#rust}

Protobuf Rust is an implementation of protocol buffers designed to be able to sit on top of other existing protocol buffer implementations that we refer to as 'kernels'.

The decision to support multiple non-Rust kernels has significantly influenced our public API, including the choice to use custom types like ProtoStr over Rust std types like str. See Rust Proto Design Decisions for more on this topic.

Packages {#packages}

Unlike in most other languages, the package declarations in the .proto files are not used in Rust codegen.

When rust_proto_library is used, the library will correspond to one crate. The name of the target is used as the crate name. Choose your library name correspondingly.

When using Cargo, we recommend you include! the generated.rs entry point in a mod of an appropriate name, as in our Example Crate.

Messages {#messages}

Given the message declaration:

message Foo {}

The compiler generates a struct named Foo. The Foo struct defines the following associated functions and methods:

Associated Functions

  • fn new() -> Self: Creates a new instance of Foo.

Traits

For a number of reasons, including gencode size, name collision problems, and gencode stability, most common functionality on messages is implemented on traits instead of as inherent implementations.

Most users should import our prelude, which only includes traits and our proto! macro and no other types (use protobuf::prelude::*). If you would rather avoid preludes, you can always import the specific traits as needed (see the

documentation here for the names and definitions of the traits if you want to import them directly).

  • fn parse(data: &[u8]) -> Result<Self, ParseError>: Parses a new instance of a message.
  • fn parse_dont_enforce_required(data: &[u8]) -> Result<Self, ParseError>: Same as parse but does not fail on missing proto2 required fields.
  • fn clear(&mut self): Clears message.
  • fn clear_and_parse(&mut self, data: &[u8]) -> Result<(), ParseError>: Clearing and parsing into an existing instance.
  • fn clear_and_parse_dont_enforce_required(&mut self, data: &[u8]) -> Result<(), ParseError>: Same as parse but does not fail on missing proto2 required fields.
  • fn serialize(&self) -> Result<Vec<u8>, SerializeError>: Serializes the message to Protobuf wire format. Serialization can fail but rarely will. Failure reasons include if the representation exceeds the maximum encoded message size (must be less than 2 GiB), and required fields (proto2) that are unset.
  • fn take_from(&mut self, other): Moves other into self, discarding any previous state that self contained.
  • fn copy_from(&mut self, other): Copies other into self, discarding any previous state that self contained. other is unmodified.
  • fn merge_from(&mut self, other): Merges other into self.
  • fn as_view(&self) -> FooView<'_>: Returns an immutable handle (view) to Foo. This is further covered in the section on proxy types.
  • fn as_mut(&mut self) -> FooMut<'_>: Returns a mutable handle (mut) to Foo. This is further covered in the section on proxy types.

Foo additionally implements the following std traits:

  • std::fmt::Debug
  • std::default::Default
  • std::clone::Clone
  • std::marker::Send
  • std::marker::Sync

Fluently Create New Instances {#proto-macro}

The API design of setters follows our established Protobuf idioms, but the verbosity when constructing new instances is a mild pain point in certain other languages. To mitigate this, we offer a proto! macro, which can be used to more-succinctly/fluently create new instances.

For example, instead of writing this:

let mut msg = SomeMsg::new();
msg.set_x(1);
msg.set_y("hello");
msg.some_submessage_mut().set_z(42);

This macro can be used to write it as follows:

let msg = proto!(SomeMsg {
  x: 1,
  y: "hello",
  some_submsg: SomeSubmsg {
    z: 42
  }
});

Message Proxy Types {#message-proxy-types}

For a number of technical reasons, we have chosen to avoid using native Rust references (&T and &mut T) in certain cases. Instead, we need to express these concepts using types - Views and Muts. These situations are shared and mutable references to:

  • Messages
  • Repeated fields
  • Map fields

For example, the compiler emits structs FooView<'a> and FooMut<'msg> alongside Foo. These types are used in place of &Foo and &mut Foo, and they behave the same as native Rust references in terms of borrow checker behavior. Just like native borrows, Views are Copy and the borrow checker will enforce that you can either have any number of Views or at most one Mut live at a given time.

For the purposes of this documentation, we focus on describing all methods emitted for the owned message type (Foo). A subset of these functions with &self receiver will also be included on the FooView<'msg>. A subset of these functions with either &self or &mut self will also be included on the FooMut<'msg>.

To create an owned message type from a View / Mut type call to_owned(), which creates a deep copy.

See the corresponding section in our design decisions documentation for more discussion about why this choice was made.

Nested Types {#nested-types}

Given the message declaration:

message Foo {
  message Bar {
      enum Baz { ... }
  }
}

In addition to the struct named Foo, a module named foo is created to contain the struct for Bar. And similarly a nested module named bar to contain the deeply nested enum Baz:

pub struct Foo {}

pub mod foo {
   pub struct Bar {}
   pub mod bar {
      pub struct Baz { ... }
   }
}

Fields {#fields}

In addition to the methods described in the previous section, the protocol buffer compiler generates a set of accessor methods for each field defined within the message in the .proto file.

Following Rust style, the methods are in lower-case/snake-case, such as has_foo() and clear_foo(). Note that the capitalization of the field name portion of the accessor maintains the style from the original .proto file, which in turn should be lower-case/snake-case per the .proto file style guide.

Fields with Explicit Presence {#explicit-presence}

Explicit presence means that a field distinguishes between the default value and no value set. In proto2, optional fields have explicit presence. In proto3, only message fields and oneof or optional fields have explicit presence. Presence is set using the features.field_presence option in editions.

Numeric Fields {#numeric-fields}

For this field definition:

int32 foo = 1;

The compiler generates the following accessor methods:

  • fn has_foo(&self) -> bool: Returns true if the field is set.
  • fn foo(&self) -> i32: Returns the current value of the field. If the field is not set, it returns the default value.
  • fn foo_opt(&self) -> protobuf::Optional<i32>: Returns an optional with the variant Set(value) if the field is set or Unset(default value) if it's unset. See Optional rustdoc.
  • fn set_foo(&mut self, val: i32): Sets the value of the field. After calling this, has_foo() will return true and foo() will return value.
  • fn clear_foo(&mut self): Clears the value of the field. After calling this, has_foo() will return false and foo() will return the default value.

For other numeric field types (including bool), int32 is replaced with the corresponding Rust type according to the scalar value types table.

String and Bytes Fields {#string-byte-fields}

For these field definitions:

string foo = 1;
bytes foo = 1;

The compiler generates the following accessor methods:

  • fn has_foo(&self) -> bool: Returns true if the field is set.
  • fn foo(&self) -> &protobuf::ProtoStr: Returns the current value of the field. If the field is not set, it returns the default value. See ProtoStr rustdoc.
  • fn foo_opt(&self) -> protobuf::Optional<&ProtoStr>: Returns an optional with the variant Set(value) if the field is set or Unset(default value) if it's unset.
  • fn set_foo(&mut self, val: impl IntoProxied<ProtoString>): Sets the value of the field. &str, String, &ProtoStr and ProtoString all implelement IntoProxied and can be passed to this method.
  • fn clear_foo(&mut self): Clears the value of the field. After calling this, has_foo() will return false and foo() will return the default value.

For fields of type bytes the compiler will generate the ProtoBytes type instead.

Enum Fields {#enum-fields}

Given this enum definition in any proto syntax version:

enum Bar {
  BAR_UNSPECIFIED = 0;
  BAR_VALUE = 1;
  BAR_OTHER_VALUE = 2;
}

The compiler generates a struct where each variant is an associated constant:

#[derive(Clone, Copy, PartialEq, Eq, Hash)]
#[repr(transparent)]
pub struct Bar(i32);

impl Bar {
  pub const Unspecified: Bar = Bar(0);
  pub const Value: Bar = Bar(1);
  pub const OtherValue: Bar = Bar(2);
}

For this field definition:

Bar foo = 1;

The compiler generates the following accessor methods:

  • fn has_foo(&self) -> bool: Returns true if the field is set.
  • fn foo(&self) -> Bar: Returns the current value of the field. If the field is not set, it returns the default value.
  • fn foo_opt(&self) -> Optional<Bar>: Returns an optional with the variant Set(value) if the field is set or Unset(default value) if it's unset.
  • fn set_foo(&mut self, val: Bar): Sets the value of the field. After calling this, has_foo() will return true and foo() will return value.
  • fn clear_foo(&mut self): Clears the value of the field. After calling this, has_foo() will return false and foo() will return the default value.

Embedded Message Fields {#embedded-message-fields}

Given the message type Bar from any proto syntax version:

message Bar {}

For any of these field definitions:

message MyMessage {
  Bar foo = 1;
}

The compiler will generate the following accessor methods:

  • fn foo(&self) -> BarView<'_>: Returns a view of the current value of the field. If the field is not set it returns an empty message.
  • fn foo_mut(&mut self) -> BarMut<'_>: Returns a mutable handle to the current value of the field. Sets the field if it is not set. After calling this method, has_foo() returns true.
  • fn foo_opt(&self) -> protobuf::Optional<BarView>: If the field is set, returns the variant Set with its value. Else returns the variant Unset with the default value.
  • fn set_foo(&mut self, value: impl protobuf::IntoProxied<Bar>): Sets the field to value. After calling this method, has_foo() returns true.
  • fn has_foo(&self) -> bool: Returns true if the field is set.
  • fn clear_foo(&mut self): Clears the field. After calling this method has_foo() returns false.

Fields with Implicit Presence (proto3 and Editions) {#implicit-presence}

Implicit presence means that a field does not distinguish between the default value and no value set. In proto3, fields have implicit presence by default. In editions, you can declare a field with implicit presence by setting the field_presence feature to IMPLICIT.

Numeric Fields {#implicit-numeric-fields}

For these field definitions:

// proto3
int32 foo = 1;

// editions
message MyMessage {
  int32 foo = 1 [features.field_presence = IMPLICIT];
}

The compiler generates the following accessor methods:

  • fn foo(&self) -> i32: Returns the current value of the field. If the field is not set, it returns 0.
  • fn set_foo(&mut self, val: i32): Sets the value of the field.

For other numeric field types (including bool), int32 is replaced with the corresponding Rust type according to the scalar value types table.

String and Bytes Fields {#implicit-string-byte-fields}

For these field definitions:

// proto3
string foo = 1;
bytes foo = 1;

// editions
string foo = 1 [features.field_presence = IMPLICIT];
bytes bar = 2 [features.field_presence = IMPLICIT];

The compiler will generate the following accessor methods:

  • fn foo(&self) -> &ProtoStr: Returns the current value of the field. If the field is not set, returns the empty string/empty bytes. See ProtoStr rustdoc.
  • fn set_foo(&mut self, value: IntoProxied<ProtoString>): Sets the field to value.

For fields of type bytes the compiler will generate the ProtoBytes type instead.

Singular String and Bytes Fields with Cord Support {#singular-string-bytes}

[ctype = CORD] enables bytes and strings to be stored as an absl::Cord in C++ Protobufs. absl::Cord currently does not have an equivalent type in Rust . Protobuf Rust uses an enum to represent a cord field:

enum ProtoStringCow<'a> {
  Owned(ProtoString),
  Borrowed(&'a ProtoStr)
}

In the common case, for small strings, an absl::Cord stores its data as a contiguous string. In this case cord accessors return ProtoStringCow::Borrowed. If the underlying absl::Cord is non-contiguous, the accessor copies the data from the cord into an owned ProtoString and returns ProtoStringCow::Owned. The ProtoStringCow implements Deref<Target=ProtoStr>.

For any of these field definitions:

optional string foo = 1 [ctype = CORD];
string foo = 1 [ctype = CORD];
optional bytes foo = 1 [ctype = CORD];
bytes foo = 1 [ctype = CORD];

The compiler generates the following accessor methods:

  • fn my_field(&self) -> ProtoStringCow<'_>: Returns the current value of the field. If the field is not set, returns the empty string/empty bytes.
  • fn set_my_field(&mut self, value: IntoProxied<ProtoString>): Sets the field to value. After calling this function foo() returns value and has_foo() returns true.
  • fn has_foo(&self) -> bool: Returns true if the field is set.
  • fn clear_foo(&mut self): Clears the value of the field. After calling this, has_foo() returns false and foo() returns the default value. Cords have not been implemented yet.

For fields of type bytes the compiler generates the ProtoBytesCow type instead.

The compiler generates the following accessor methods:

  • fn foo(&self) -> &ProtoStr: Returns the current value of the field. If the field is not set, returns the empty string/empty bytes.
  • fn set_foo(&mut self, value: impl IntoProxied<ProtoString>): Sets the field to value.

Enum Fields {#implicit-presence-enum}

Given the enum type:

enum Bar {
  BAR_UNSPECIFIED = 0;
  BAR_VALUE = 1;
  BAR_OTHER_VALUE = 2;
}

The compiler generates a struct where each variant is an associated constant:

#[derive(Clone, Copy, PartialEq, Eq, Hash)]
#[repr(transparent)]
pub struct Bar(i32);

impl Bar {
  pub const Unspecified: Bar = Bar(0);
  pub const Value: Bar = Bar(1);
  pub const OtherValue: Bar = Bar(2);
}

For these field definitions:

// proto3
Bar foo = 1;

// editions
message MyMessage {
 Bar foo = 1 [features.field_presence = IMPLICIT];
}

The compiler will generate the following accessor methods:

  • fn foo(&self) -> Bar: Returns the current value of the field. If the field is not set, it returns the default value.
  • fn set_foo(&mut self, value: Bar): Sets the value of the field. After calling this, has_foo() will return true and foo() will return value.

Repeated Fields {#repeated-fields}

For any repeated field definition the compiler will generate the same three accessor methods that deviate only in the field type.

In editions, you can control the wire format encoding of repeated primitive fields using the repeated_field_encoding feature.

// proto2
repeated int32 foo = 1; // EXPANDED by default

// proto3
repeated int32 foo = 1; // PACKED by default

// editions
repeated int32 foo = 1 [features.repeated_field_encoding = PACKED];
repeated int32 bar = 2 [features.repeated_field_encoding = EXPANDED];

Given any of the above field definitions, the compiler generates the following accessor methods:

  • fn foo(&self) -> RepeatedView<'_, i32>: Returns a view of the underlying repeated field. See RepeatedView rustdoc.
  • fn foo_mut(&mut self) -> RepeatedMut<'_, i32>: Returns a mutable handle to the underlying repeated field. See RepeatedMut rustdoc.
  • fn set_foo(&mut self, src: impl IntoProxied<Repeated<i32>>): Sets the underlying repeated field to a new repeated field provided in src. Accepts a RepeatedView, RepeatedMut or Repeated. See Repeated rustdoc.

For different field types only the respective generic types of the RepeatedView, RepeatedMut and Repeated types will change. For example, given a field of type string the foo() accessor would return a RepeatedView<'_, ProtoString>.

Map Fields {#map}

For this map field definition:

map<int32, int32> weight = 1;

The compiler will generate the following 3 accessor methods:

  • fn weight(&self) -> protobuf::MapView<'_, i32, i32>: Returns an immutable view of the underlying map. See MapView rustdoc.
  • fn weight_mut(&mut self) -> protobuf::MapMut<'_, i32, i32>: Returns a mutable handle to the underlying map. See MapMut rustdoc.
  • fn set_weight(&mut self, src: protobuf::IntoProxied<Map<i32, i32>>): Sets the underlying map to src. Accepts a MapView, MapMut or Map. See Map rustdoc.

For different field types only the respective generic types of the MapView, MapMut and Map types will change. For example, given a field of type string the foo() accessor would return a MapView<'_, int32, ProtoString>.

Any {#any}

Any is not special-cased by Rust Protobuf at this time; it will behave as though it was a simple message with this definition:

message Any {
  string type_url = 1;
  bytes value = 2;
}

Oneof {#oneof}

Given a oneof definition like this:

oneof example_name {
    int32 foo_int = 4;
    string foo_string = 9;
    ...
}

The compiler will generate accessors (getters, setters, hazzers) for every field as if the same field was declared as an optional field outside of the oneof. So you can work with oneof fields like regular fields, but setting one will clear the other fields in the oneof block. In addition, the following types are emitted for the oneof block:

  #[non_exhaustive]
  #[derive(Debug, Clone, Copy)]

  pub enum ExampleNameOneof<'msg> {
    FooInt(i32) = 4,
    FooString(&'msg protobuf::ProtoStr) = 9,
    not_set(std::marker::PhantomData<&'msg ()>) = 0
  }
  #[derive(Debug, Copy, Clone, PartialEq, Eq)]

  pub enum ExampleNameCase {
    FooInt = 4,
    FooString = 9,
    not_set = 0
  }

Additionally, it will generate the two accessors:

  • fn example_name(&self) -> ExampleNameOneof<_>: Returns the enum variant indicating which field is set and the field's value. Returns not_set if no field is set.
  • fn example_name_case(&self) -> ExampleNameCase: Returns the enum variant indicating which field is set. Returns not_set if no field is set.

Enumerations {#enumerations}

Given an enum definition like:

enum FooBar {
  FOO_BAR_UNKNOWN = 0;
  FOO_BAR_A = 1;
  FOO_B = 5;
  VALUE_C = 1234;
}

The compiler will generate:

  #[derive(Clone, Copy, PartialEq, Eq, Hash)]
  #[repr(transparent)]
  pub struct FooBar(i32);

  impl FooBar {
    pub const Unknown: FooBar = FooBar(0);
    pub const A: FooBar = FooBar(1);
    pub const FooB: FooBar = FooBar(5);
    pub const ValueC: FooBar = FooBar(1234);
  }

Note that for values with a prefix that matches the enum, the prefix will be stripped; this is done to improve ergonomics. Enum values are commonly prefixed with the enum name to avoid name collisions between sibling enums (which follow the semantics of C++ enums where the values are not scoped by their containing enum). Since the generated Rust consts are scoped within the impl, the additional prefix, which is beneficial to add in .proto files, would be redundant in Rust.

Extensions {#extensions}

Protobuf Rust supports extensions for proto2 messages. Extensions are accessed via generated ExtensionId constants.

Given a message with extensions defined:

package xyz;
message Foo {
  extensions 100 to 199;
}

extend Foo {
  optional int32 i32_ext = 100;
  repeated int32 repeated_i32_ext = 101;
}

The compiler generates constants of type proto::ExtensionId named I32_EXT and REPEATED_I32_EXT, which you can use to read and write the extensions on the Foo type.

Accessing Extensions {#accessing-exts}

The ExtensionId type provides methods to interact with extensions on a message. These methods work with owned messages, views, and muts.

  • fn has(&self, msg: impl AsView<Target = M>) -> bool: Returns true if the extension is set on the message. (Not available for repeated extensions, by design).
  • fn get(&self, msg: impl AsView<Target = M>) -> View<'_, E>: Returns the value of the extension. If not set, returns the default value. For repeated fields, returns a RepeatedView.
  • fn set(&self, msg: impl AsMut<Target = M>, value: impl IntoProxied<E>): Sets the value of the extension.
  • fn clear(&self, msg: impl AsMut<Target = M>): Clears the extension from the message.
  • fn get_mut(&self, msg: impl AsMut<Target = M>) -> Mut<'_, E>: Returns a mutable handle to the extension. For messages and repeated fields, this will create the field if it doesn't exist.

Example Usage {#ext-example}

use protobuf::prelude::*;

let mut foo = xyz::Foo::new();

// Check and set scalar extension
assert!(!xyz::INT32_EXT.has(&foo));
xyz::INT32_EXT.set(&mut foo, 42);
assert!(xyz::INT32_EXT.has(&foo));
assert_eq!(xyz::INT32_EXT.get(&foo), 42);

// Clear scalar extension
xyz::INT32_EXT.clear(&mut foo);
assert!(!xyz::INT32_EXT.has(&foo));

// Working with repeated extensions
{
    let mut rep_mut = xyz::REPEATED_INT32_EXT.get_mut(&mut foo);
    rep_mut.push(1);
    rep_mut.push(2);
}
assert_eq!(xyz::REPEATED_INT32_EXT.get(&foo).len(), 2);
assert_eq!(xyz::REPEATED_INT32_EXT.get(&foo).get(0), Some(1));

Arena Allocation {#arena}

A Rust API for arena allocated messages has not yet been implemented.

Internally, Protobuf Rust on upb kernel uses arenas, but on C++ kernels it doesn't. However, references (both const and mutable) to messages that were arena allocated in C++ can be safely passed to Rust to be accessed or mutated.

Services {#services}

A Rust API for services has not yet been implemented.