Serialization

The data in your message has been set and is ready to be transmitted over your communication line. So how do you serialize the message?

Each message object has a function serialize() derived from one of its base classes. When calling this function, the data in the message is serialized using the Protocol Buffers format. This function is similar to the Protobuf function SerializeToString() of the C++ implementation.

However, there is a difference between the two. Where SerializeToString() outputs into a std::string. Strings are not suitable for embedded implementations. The standard std::string used dynamic memory allocation, which we are trying to avoid.

The interface class WriteBufferInterface is the alternative to using std::string. The object is referenced to the message serialize() function, where it is used to store the output of the serialization. But as the name suggests, it is an interface. The actual implementation which you use is the WriteBufferFixedSize template class. Specify the number of bytes allocated in the class via the template parameter:

::EmbeddedProto::WriteBufferFixedSize<64> buffer;

auto serialization_status = msg.serialize(buffer);
if(::EmbeddedProto::Error::NO_ERRORS == serialization_status)
{
  // buffer.get_data() and buffer.get_size() hold the serialized message.
}

Prior to version 3.2.0, it was up to the user of this library to write an implementation of the WriteBufferInterface specific to their situation. Implementations could be a simple array, or you could write the data directly to a UART bus. The interface is simple and easily implemented. You can find more documentation on the required functions here.

Please note that the interface has two push() functions, one for a single byte and one for a block of bytes. As of version 4.0.0 the library writes a repeated field of fixed-width numbers as one block. Your implementation of push(const const_bytes_view& bytes) should copy the block in one go rather than loop over the single byte version, or the gain is lost. It must also append the whole block or nothing at all. See packed fixed-width fields for the guarantees a buffer of your own has to give.

The size of a message

How large should the buffer be? Two functions answer this question. The function serialized_size() returns the number of bytes the message takes with the values it holds at that moment. The static function max_serialized_size() returns the largest number of bytes the message can ever take. It is a constant expression, so you can use it to size your buffer at compile time:

::EmbeddedProto::WriteBufferFixedSize<Foo::max_serialized_size()> buffer;

Please note that a message with a callback field has no maximum size. For such a message max_serialized_size() returns UINT32_MAX.

Serializing in parts

Sometimes the buffer you can afford is smaller than the message. Think of a message with a large repeated field, sent over a link with small packets. As of version 4.0.0 a message can be serialized in parts, into a buffer which is emptied in between. This feature is switched on by defining PARTIAL_SERIALIZATION_ENABLED when building. It is off by default, as it adds code to every message.

With the define set, each message has a function serialize_partial(). It takes the buffer and a state object which remembers where serialization stopped. The generated message provides a type for the state, sized to the nesting depth of the message. Call the function until it returns NO_ERRORS. Each time it returns BUFFER_FULL, transmit the buffer, clear it, and call again:

::EmbeddedProto::WriteBufferFixedSize<32> buffer;
Foo::StateStack state;

auto status = msg.serialize_partial(buffer, state.root());
while(::EmbeddedProto::Error::BUFFER_FULL == status)
{
  transmit(buffer.get_data(), buffer.get_size());
  buffer.clear();
  status = msg.serialize_partial(buffer, state.root());
}

if(::EmbeddedProto::Error::NO_ERRORS == status)
{
  // Transmit the last part.
  transmit(buffer.get_data(), buffer.get_size());
}

The buffer must at least hold the largest single scalar, ten bytes. The message on the wire is identical to one serialized in a single call. The receiving side can use deserialize() as usual, or receive it in parts as well, see deserialization.