Protobuf In C# .NET – Part 2 – Serializing/Deserializing

This is a 4 part series on working with Protobuf in C# .NET. While you can start anywhere in the series, it’s always best to start at the beginning!

Part 1 – Getting Started
Part 2 – Serializing/Deserializing
Part 3 – Using Length Prefixes
Part 4 – Performance Comparisons

In our last post, we spent much of the time talking about how proto contracts work. But obviously that’s all for nothing if we don’t start serializing some data. Thankfully for us, the Protobuf.NET library takes almost all of the leg work out of it, and we more or less follow the same paradigms that we did when working with XML or JSON in C#.

Of course, if you haven’t already, install Protobuf.NET into your application using the following package manager console command :

Install-Package protobuf-net

I’m going to be using the same C# contract we used in the last post. But for reference, here it is again.

[ProtoContract]
class Person
{
    [ProtoMember(1)]
    public string FirstName { get; set; }

    [ProtoMember(2)]
    public string LastName { get; set; }

    [ProtoMember(3)]
    public List Emails { get; set; }
}

And away we go!

Serializing Data

To serialize or write our data in protobuf format, we simply need to take our object and push it into a stream. An in memory example (For example if you needed a byte array to send somewhere else), would look like this :

var person = new Person
{
    FirstName = "Wade",
    LastName = "Smith",
    Emails = new List
    {
        "wade.smith@gmail.com", 
        "wsmith@company.com"
    }
};

using(var memoryStream = new MemoryStream())
{
    Serializer.Serialize(memoryStream, person);
    var byteArray = memoryStream.ToArray();
}

So ignoring our set up code there for the Person object, we’ve basically serialized in 1 or 5 lines of code depending on if you want to count the setup of the memory stream. Pretty trivial and it makes all that talk about Protobuf being some sort of voodoo really just melt away.

If we wanted to, we could instead serialize directly to a file like so :

using (var fileStream = File.Create("person.buf"))
{
    Serializer.Serialize(fileStream, person);
}

This leaves us with a person.buf file locally. Of course, if we open this file in a text editor it’s unreadable (Protobuf is not human readable when serialized), but we can use a tool such as https://protogen.marcgravell.com/decode to open the file and tell us what’s inside of it.

Doing that, we get :

Field #1: 0A String Length = 4, Hex = 04, UTF8 = “Wade”
Field #2: 12 String Length = 5, Hex = 05, UTF8 = “Smith”
Field #3: 1A String Length = 20, Hex = 14, UTF8 = “wade.smith@gmail …” (total 20 chars)
Field #3: 1A String Length = 18, Hex = 12, UTF8 = “wsmith@company.c …” (total 18 chars)

Notice that the fields within our protobuf file are identified by their integer identifer, *not* by their string property name. Again, this is important to understand because we need the same proto contract identifiers on both ends to know that Field 1 is actually a persons first name.

Well that’s serialization done, how about deserializing?

Deserializing Data

Of course if serializing data can be done in 1 line of code, deserializing or reading back the data is going to be just as easy.

using (var fileStream = File.OpenRead("person.buf"))
{
    var myPerson = Serializer.Deserialize<Person>(fileStream);
    Console.WriteLine(myPerson.FirstName);
}

This is us simply reading a file and deserializing it into our myPerson object. It’s somewhat trivial and really straight forward if I’m being honest and there actually isn’t too much to deep dive into.

That is.. Until we start talking about length prefixes. Length prefixes are protobufs way of serializing several piece of data into the same data stream. So imagine that if we have 5 people, how can we store 5 people in the same file or data stream and know when one persons data ends, and another begins. In the next part of this series we’ll be taking a look at just how that works with Protobuf.NET! Check it out : https://dotnetcoretutorials.com/2022/01/14/protobuf-in-c-net-part-3-using-length-prefixes/