Yandex has released YaFF (Yet Another Flat Format) as an open-source zero-copy wire format for the Protobuf ecosystem, promising read speeds that approach those of raw C++ structs—without abandoning the .proto schema as the single source of truth. Under the Apache 2.0 license and written in C++, YaFF v0.1.0 addresses a specific pain point: Protobuf parsing can devour double-digit percentages of CPU in high-load backends, while existing zero-copy alternatives such as FlatBuffers require maintaining a separate schema and break semantic compatibility with Protobuf. YaFF sits in the gap, offering a drop-in alternative that preserves Protobuf semantics and lets developers migrate one module at a time.
What Is YaFF and Why It Matters
YaFF is not a replacement for Protobuf—it is an alternative wire format for Protobuf messages. The same .proto file generates a Protobuf-like C++ API, but reads require no parsing step; fields are accessed directly from the buffer. For less performance-sensitive paths, two-way conversion lets teams keep Protobuf everywhere else. This incremental approach means you introduce YaFF only in the hot path that actually needs the speed. At Yandex’s scale, the savings are tangible: the company reports a 10–20% CPU reduction in its advertising recommendation system.
The Problem: Protobuf Parsing as a Hot-Path Bottleneck
In high-load server backends, parsing Protobuf messages can consume a large fraction of CPU time—enough to map to thousands of physical cores at Yandex’s scale. The industry’s common zero-copy answer, FlatBuffers, sidesteps parsing but requires its own schema and a separate conversion layer, making it semantically incompatible with Protobuf. Teams often conclude that the migration effort—duplicated schemas, different evolution rules, hand-written converters—is not worth the performance gain. YaFF aims at exactly that gap: zero-copy reads with Protobuf semantics intact.
The Four Layouts: Fixed, Flat, Sparse, and Dynamic
A layout in YaFF determines how a message is stored in the buffer, changing only the physical representation while leaving the schema and generated interfaces unchanged. YaFF ships four layouts, each with different trade-offs between read speed and schema flexibility:
- Fixed – A plain packed struct with no header and a frozen schema. Best for small inlined primitives.
- Flat – Adds a two-byte header and supports restricted schema evolution (type preservation). Best for dense, hot data.
- Sparse – Addresses fields through a meta table, adding six bytes overhead but allowing unrestricted schema evolution. Best for sparse schemas.
- Dynamic (default) – Selects Flat or Sparse at runtime based on schema evolution. Uses Flat while the schema permits, then switches to Sparse when changes break flat alignment.
Benchmarks: Near-Struct Read Speeds
Yandex provides a reproducible benchmark suite using google/benchmark on an AMD EPYC 7713 with Clang 20.1.8. In the hot hierarchical test case, median read times show a clear hierarchy:
| Format | Read time (ns) | Slowdown vs raw struct |
| Raw C++ struct | 8.14 | 1.0x |
| YaFF Flat Layout | 9.79 | 1.2x |
| YaFF Sparse Layout | 21.23 | 2.6x |
| FlatBuffers | 37.30 | 4.6x |
| Protobuf | 219.35 | 26.9x |
The Flat Layout reads about 3.8x faster than FlatBuffers and 22x faster than Protobuf, staying within 1.2x of the raw struct. Absolute numbers depend on hardware, but the ratios are expected to hold across platforms.
How YaFF Avoids the Aliasing Problem
Both FlatBuffers and YaFF read fields by reinterpreting raw memory as the target type. This type-punning often confuses LLVM’s alias analysis, which falls back to a conservative MayAlias verdict, forcing repeated re-walks of the access chain. YaFF’s generated code includes annotations that tell the compiler when reuse is safe. As long as no writes occur between reads, YaFF caches the access chain—eliminating the overhead that FlatBuffers incurs in deep nested access patterns.
Practical Use Cases and Code Walkthrough
YaFF is designed for systems where you control both producer and consumer. Recommendation and ad-serving backends, memory-mapped indexes, search indexes, feature stores, and feed services all fit. The planned Columnar Layout will target analytics and ML pipelines with large repeated fields. The API mirrors Protobuf but removes the parse step:
#include "feed.pb.h" // generated by protoc
#include "feed.yaff.h" // generated by yaff_generate()
// Serialize an existing Protobuf message into a YaFF buffer.
feed::FeedResponse proto = LoadFeedResponse();
const auto buffer = yaff::Serialize<feed::FeedResponse>(proto);
// Read fields directly from the buffer. No parsing step.
const auto& response = yaff::ReadMessage<feed::FeedResponse>(buffer.Data());
for (const auto& item : response.items()) {
std::string_view title = item.title();
std::string_view author = item.author().name(); // empty if unset
}
// Convert back to Protobuf when needed.
feed::FeedResponse restored;
response.ParseTo(restored);
prepreprepre
Integration uses CMake (find_packagecodecodecodecode) or Conan. After running protobuf_generate()codecodecodecode, call yaff_generate()codecodecodecode. Generated types live in the protoyaff::<package>codecodecodecode namespace. Most projects only need to link yaff::corecodecodecodecode and yaff::protocodecodecodecode.
Who Should Try This Now
If your team maintains a Protobuf-based service that spends measurable CPU time on deserialization—especially in read-heavy, latency-sensitive hot paths—YaFF offers a low-risk, high-reward experiment. You can drop it into one module, with two-way conversion at the boundaries, and measure the CPU savings directly. The project is at v0.1.0 with C++ support only, but the benchmarks and production deployment at Yandex suggest a mature foundation. Developers interested in zero-copy serialization without abandoning Protobuf’s ecosystem should evaluate YaFF on their own hardware using the provided benchmark suite. The repository and documentation are available at GitHub and yaff.tech.
