Repeated fields

Repeated fields

In microcontrollers, dynamic memory allocation might cause problems. When running bare metal code it is hard to catch exceptions caused by insufficient memory and other related errors. Repeated fields in Protobuf are by default of unknown length. Their implementations thus often make use of dynamic memory allocation.

In Embedded Proto this is not the case. Repeated fields are implemented as arrays with a static length. The array size is passed to the message by means of a template parameter (option 1) or through a custom option in the .proto file (option 2). We will discuss both in the following paragraphs.

Option 1: Set the length using a template parameter

As an example we take the following .proto message definition:

message Foo
{
  repeated uint32 y = 1;
}

The class this will generate a template parameter with which you can set the size of the message. The definition of the class and the array object for variable y looks somewhat like this:

template<uint32_t y_SIZE> class Foo
{
  private:
     ::EmbeddedProto::RepeatedFieldSize<EmbeddedProto::uint32, y_SIZE> y_;
};

Option 2: Set the length in the .proto file

As of version 3.0.0, it is also possible to fix the length of a repeated field in the .proto file. This also applies to String and Bytes fields. You will not get the template option but an array of the size you specify in your .proto file. This is done by using a mechanism called custom options provided by Google Protobuf. There are two advantages to doing this:

1. It will reduce the number of template parameters in the code.
2. You could use the custom options to communicate the field length to other languages. Allowing them to know the limits of the embedded code. Read up on custom options if you wish to know how to do this.

The first thing when using custom options is to make sure you have run the setup.py script correctly after updating Embedded Proto. This is required as an options file is generated during this process. The generated code is required by the Embedded Proto plugin to interpret the options. To keep the generated code up to date it is required to rerun the script each time you update Embedded Proto. Please refer to the installation manual or readme file on how to do this.

Next, let us modify your .proto files to include two additional elements:

import "embedded_proto_options.proto"; // Number 1

message Foo
{
  repeated uint32 y = 1 [(EmbeddedProto.options).maxLength = 10]; // Number 2
}

Number 1: Include the definitions of the Embedded Proto custom options file. This file is provided in the Embedded Proto repo in the ./generator folder. This is the file for which source code is generated with the setup script.

Number 2: Add the custom option to each repeated field for which you wish to set the length. In this example, the length is set to 10 elements.

Finally, when generating the code based on this .proto file you need to include the folder with the embedded_proto_options.proto file. Please note the additional -I./generator in the command below:

protoc --plugin=protoc-gen-eams=protoc-gen-eams -I./LOCATION/PROTO/FILES -I./generator --eams_out=./generated_src PROTO_MESSAGE_FILE.proto

The generated code will now have the length filed in. So where in the templated option you will see something like y_SIZE, when using the custom option this will be replaced by a fixed number.

Usage of repeated fields

To access the array and the elements in the repeated field various functions are generated in the message.

void add_y(const EmbeddedProto::uint32& value);
void set_y(uint32_t index, const EmbeddedProto::uint32& value);
void set_y(uint32_t index, const EmbeddedProto::uint32&& value);

::EmbeddedProto::RepeatedFieldSize<EmbeddedProto::uint32, y_SIZE>& mutable_y();

const EmbeddedProto::uint32& y(uint32_t index) const;
::EmbeddedProto::RepeatedFieldSize<EmbeddedProto::uint32, y_SIZE>& get_y() const;

void clear_y();

Setting values in the array can be done by adding data to the end of the array with add_y() or at a given index with set_y(). It is also possible to set the data via the mutable function mutable_y(). This function returns the whole array as a non-constant reference.

This function could also be used to just retrieve data but it is better to use the const version in that case by calling get_y(). Accessing individual elements is also possible using the function y(). Please note that you have to check yourself if your index is within the bounds of the array size.

Finally, you can clear the whole array with the clear_y() function.