Protobuf 101: How to Build Faster Applications with Structured Binary Data


Protobuf 101: How to Build Faster Applications with Structured Binary Data

protocolbuffers/protobuf

2026-01-08

Here is a breakdown of what Protocol Buffers (or Protobuf) is, why we love it, and how you can get started.

In the world of software, different services often need to talk to each other. Usually, we use JSON because it's easy for humans to read. However, JSON is "wordy" and slow for computers to process.

Protobuf is Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data. Unlike JSON, it is binary, making it much smaller and faster.

Performance
It’s significantly faster to serialize (pack) and deserialize (unpack) than JSON or XML.

Type Safety
You define your data structure strictly. No more guessing if a field is a string or an integer.

Backward Compatibility
You can add new fields to your data format without breaking "old" programs that don't know about them yet.

Code Generation
You write a simple .proto file, and Protobuf generates the classes for you in Java, Python, C++, Go, and more.

First, you’ll need the protoc compiler.

Mac
brew install protobuf

Ubuntu
sudo apt install -y protobuf-compiler

Windows
Download the pre-built binary from the GitHub releases page.

You start by creating a schema. Let's say we are building a user profile system. Create a file named user.proto

syntax = "proto3";

package my_app;

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
  repeated string roles = 4; // This is a list/array
}

Run the compiler to generate code for your favorite language (e.g., Python)

protoc --python_out=. user.proto

Once the code is generated, using it in your application is incredibly simple.

import user_pb2

# Create a new user
user = user_pb2.User()
user.id = 123
user.name = "Alice"
user.email = "[email protected]"
user.roles.append("admin")

# Serialize to a binary string to send over the network or save to a file
binary_data = user.SerializeToString()
print(f"Binary data size: {len(binary_data)} bytes")

# Imagine receiving 'binary_data' from a server
new_user = user_pb2.User()
new_user.ParseFromString(binary_data)

print(f"ID: {new_user.id}")
print(f"Name: {new_user.name}")

While Protobuf is powerful, it's not always the right choice. Use it when

Microservices
When many internal services are talking to each other (gRPC uses Protobuf by default).

Mobile Apps
To save data usage for users on slow connections.

Large Scale
When you are processing millions of messages and need to save on CPU and bandwidth.

Engineer's Tip
Stick to JSON for public-facing APIs where you want third-party developers to easily test things in their browser. Use Protobuf for everything "under the hood" where speed is king!


protocolbuffers/protobuf