About Buf

Buf's goal is to shift API development toward a schema-driven paradigm and thus pave the way for a future in which APIs are defined in a way that service owners and clients can depend on.

Defining APIs using an IDL provides a number of benefits over simply exposing REST/JSON services, and today, Protobuf is the most stable, widely adopted IDL in the industry. But as things stand today, using Protobuf is much more difficult than using JSON as your data transfer format.

Buf is building tooling to make Protobuf reliable and user friendly for service owners and clients, while keeping it the obvious choice on the technical merits. Your organization should not have to reinvent the wheel to create, maintain, and consume Protobuf APIs efficiently and effectively. We'll handle your Protobuf management strategy for you, so you can focus on what matters.

The problems we aim to solve

Traditionally, adopting Protobuf presents a number of challenges across the API lifecycle. These are the problems we aim to solve:

API designs are often inconsistent: Writing maintainable, consistent Protobuf APIs isn't as widely understood as writing maintainable REST/JSON-based APIs. With no standards enforcement, inconsistency can arise across an organization's Protobuf APIs, and design decisions can inadvertently affect your API's future iterability.
Dependency management is usually an afterthought: Protobuf files are vendored manually, with an error-prone copy-and-paste process from GitHub repositories. Before the BSR, there was no centralized attempt to track and manage around cross-file dependencies. This is analogous to writing JavaScript without npm, Rust without cargo, Go without modules, and all of the other programming language dependency managers we've all grown so accustomed to.
Forwards and backwards compatibility is not enforced: While forwards and backwards compatibility is a promise of Protobuf, actually maintaining backwards-compatible Protobuf APIs isn't widely practiced, and is hard to enforce.
Stub distribution is a difficult, unsolved process: Organizations have to choose to either centralize their protoc workflow and distribute generated code, or require all service clients to run protoc independently. Because there is a steep learning curve to using protoc (and the associated protoc plugins) in a reliable manner, organizations often struggle with distributing their Protobuf files and stubs. This creates substantial overhead, and often requires a dedicated team to manage the process. Even when using a build system like Bazel, exposing APIs to external customers remains problematic.
The tooling ecosystem is limited: Many user-friendly tools exist for REST/JSON APIs today. On the other hand, mock server generation, fuzz testing, documentation, and other daily API concerns are not widely standardized or user friendly for Protobuf APIs. As a result, teams regularly reinvent the wheel and build custom tooling to replicate the JSON ecosystem.

Buf is building a modern Protobuf ecosystem

Our tools address many of the problems above, ultimately allowing you to redirect much of your time and energy from managing Protobuf files to implementing your core features and infrastructure.

The `buf` CLI

The buf CLI enables you to create consistent Protobuf APIs that preserve compatibility and comply with best practices. The tool is currently available on an open-source basis. The buf CLI incorporates these components to help you create consistent Protobuf APIs:

A new developed high-performance Protobuf compiler.
A linter that enforces good API design choices and structure.
A breaking change detector that enforces compatibility at the source code or wire level.
A generator that invokes your protoc plugins based on a configurable template.

The Buf Schema Registry (BSR)

The Buf Schema Registry (BSR) is a hosted SaaS platform that serves as your organization’s source of truth for your Protobuf APIs. The BSR enables you to centrally maintain compatibility and manage dependencies, while enabling your clients to consume APIs reliably and efficiently. Similar to npm for JavaScript, pip for Python, or cargo for Rust, the BSR finally brings dependency management to your Protobuf APIs.

Where to go from here

See the installation page to install the buf CLI.

Next, we recommend completing the tour. The tour provides an overview of most of the existing functionality of Buf and takes approximately 20 minutes to complete.

After completing the tour, check out the remainder of the documentation for your specific areas of interest. We've aimed to provide as much documentation as we can for the various components of Buf to give you a full understanding of Buf's surface area.