Why binary formats break when types change
Compact binary formats such as bincode and postcard write the field values one after another in declaration order and nothing else. This is fast and small, but the encoding can only be read by the exact type that wrote it. If you add a field, remove one or reorder them, every reader must be updated in the same release and previously written data becomes unreadable.
The usual alternatives are JSON, which tolerates change but is large, slow
and flattens Rust's data model, and Protocol Buffers, which evolve well but
bring a separate .proto schema, generated types and their own
data model. Postbag sits in between. It is a compact binary format for plain
Serde types that supports schema evolution without a schema language and
without explicit versioning.
How Postbag encodes for change
Postbag's Full format writes each field together with its
identifier and the length of its value. A reader matches fields by
identifier instead of position. Unknown fields are skipped over, missing
fields take their #[serde(default)] and the order of fields
does not matter. The same applies to enum variants, with
#[serde(other)] catching variants the reader does not know. You
can therefore add, remove, rename and reorder fields and variants while old
and new programs exchange data. Backward and forward compatibility both
hold: old programs read new data and new programs read old data.
The identifiers are normally the field names; to save space they can be
numbered without giving up evolvability. If
minimal size matters more than free evolution, the Slim format
writes the values only and still allows appending fields and variants at the
end. Both formats use variable-length integer encoding and both are
specified independently of the
implementation.
When a change is not covered
Some changes are incompatible under any rule set, for example completely
changing the type of a field. For these cases a field can be wrapped with
#[serde(with = "postbag::recoverable")]. If deserialization of
such a field fails, the failure is confined to it, the rest of the value is
recovered and the field takes its default value. Marking an existing field
as recoverable does not change its wire representation.
Questions
What happens when a reader meets a field it does not know?
In the Full format every field carries its identifier and the length of its value, so an unknown field is simply skipped. A field the reader expects but does not receive takes its #[serde(default)], and an unknown enum variant is caught by a variant marked with #[serde(other)].
Do I need a schema file or explicit version numbers?
No. Your Rust types are the schema and there is no version negotiation. Each side simply serializes its own version of the type, and the compatibility rules ensure that both sides can read each other's data.
How does this compare to bincode or postcard?
Bincode and postcard write the values only, in declaration order, so most changes to a type make old data unreadable. Postbag Slim makes the same trade-off with limited evolvability, while Postbag Full spends a few extra bytes on identifiers and lengths so that fields and variants can change freely.
What about a change the rules do not cover?
You can mark such a field as recoverable. If its type has changed incompatibly, the rest of the enclosing value is still deserialized and the field takes its default value instead of failing the whole read.
Does it preserve Rust's data model exactly?
Yes. Postbag round-trips Serde's data model without flattening it, including distinctions such as Some(None) versus None, which many self-describing formats lose.
Where to go next
The Postbag overview has a quick start and the complete compatibility rules, and file formats that survive upgrades covers the case of config and save files. Postbag is also the codec used by Remoc, where it allows communicating programs to be upgraded one at a time.