When you want to make a module available for other developers, you publish it so that it’s visible to your team,
organization, or the wider buf community. Once you’ve published the module, developers importing its packages will be
able to resolve a dependency on the module by running buf mod update
. For more information, have a look
at Dependency Management
Publishing steps
Use the following steps to publish a module from your local machine using the buf
CLI.
-
Open a command prompt and change to your module’s root directory in the local repository.
-
Run
buf mod update
, to fetch the latest digests for the specified references in the config file, and write them and their transitive dependencies to thebuf.lock
file.$ buf mod update
-
Run
buf build
a final time to make sure everything is working.This builds the Protobuf files into a Buf image which is necessary for the module to be accepted into the registry
$ buf build
-
Create a draft for the module with a draft name.
For the draft name, use a name that signals to users the nature of changes in this draft. For more, see Module versioning.
$ buf push --draft <DRAFT_NAME>
When the draft is pushed, it becomes available on the BSR, along with its documentation. Take this opportunity to validate your changes are correct. You can depend on a draft module using its reference of the form
<module-name>:<draft-name>
. For more information on drafts, see Publishing Drafts -
Once satisfied, you can now push your module to the origin repository. If successful,
buf
will output the BSR commit hash.$ buf push
Output5173e5cfeb904508839378050d95e1de
As an example, Go developers interested in your module import a package from it and run the go get
command just as
they would with any other module. They can run the go get
command for latest versions or they can specify a particular
version, as in the following example:
go get buf.build/gen/go/bufbuld/eliza/protocolbuffers/go@<version-or-draft-name>
To get a remote package that includes generated TypeScript code with protobuf-es:
npm install @buf/bufbuld_eliza.bufbuild_es@<version-or-draft-name>
Publishing drafts
The Buf Schema Registry (BSR) stores and manages Protobuf files as versioned
modules so that individuals and organizations can publish and consume their APIs without friction. However, having only
a main
commit history in the BSR makes it difficult for engineers to push work-in-progress modules for testing or
validation in the same way that they would push commits to a git feature branch. To enable engineers to begin
iterating quickly using generated code without the need to merge Protobuf schemas to main
, we support BSR drafts.
Drafts are not included in the main
commit history, can be deleted or overwritten, and work seamlessly with:
- GitHub branches: Drafts can be automatically deployed to the BSR when a change is pushed to a GitHub branch/PR by configuring a GitHub action.
- The
buf
CLI: Drafts can be used locally with thebuf
CLI and commands likebuf build
andbuf generate
. - Remote packages: Any draft can be used with remote packages, enabling in-progress changes to be integrated using
tools like
npm
andgo get
.
Creating drafts
Users can manually push drafts with a simple buf
CLI command:
buf push --draft <DRAFT_NAME>
When the draft is pushed, it becomes available on the BSR, along with its documentation. A link at the top of the repository page will show the number of drafts that were pushed to that repository and provide the ability to navigate to them:
Users can also integrate drafts by adding the buf-push-action
to
their GitHub workflows. With this integration,
engineers can automatically deploy drafts with their Protobuf definitions to the BSR whenever they push to a GitHub
branch or pull request!
To illustrate, consider this example:
name: buf-push
on: push # Apply to all pushes
jobs:
push-module:
# Run `git checkout`
- uses: actions/checkout@v3
# Install the `buf` CLI
- uses: bufbuild/buf-setup-action@v1
# Push the module to the BSR
- uses: bufbuild/buf-push-action@v1
with:
buf_token: ${{ secrets.BUF_TOKEN }}
# Push as a draft when not pushing to `main`
draft: ${{ github.ref_name != 'main' }}
The draft
flag indicates whether the module should be pushed as a draft. In the above example, the module will be
pushed to the BSR as a draft, if the GitHub push did not occur on the main
git branch. The name of the BSR draft will
be the short ref name of the git
branch or git tag that triggered the workflow run. For more details on buf-push-action
, check out
the GitHub repository and
the Buf documentation.
Updating drafts
Drafts are directly mapped to a commit, there is no concept of history in drafts. If you need to make changes to a module in draft, create a new draft.
Using drafts with the Buf CLI
To consume a draft with the buf
CLI, use the name of the draft as a module reference. For example, assuming
a buf.gen.yaml
file exists in the current directory, code can
be generated for a draft with the name <draft-name>
using:
buf generate buf.build/bufbuld/eliza:<draft-name>
The :<DRAFT_NAME>
reference can also be used in other buf
CLI commands
including buf build
, buf breaking
, buf export
, buf curl
, or as a dependency in
the buf.yaml
:
version: v1
deps:
- buf.build/bufbuld/eliza:<draft-name>
breaking:
use:
- FILE
lint:
use:
- DEFAULT
Please be aware that because drafts can be deleted and overwritten, they can only be used as dependencies locally. A module that has a dependency on a draft cannot be pushed to the BSR.
$ buf push
OutputFailure: failed to push module, pushing a module with a dependency pinned to a draft commit is not allowed, dependency "buf.build/bufbuild/eliza" is pinning to a draft commit "b331a4bb1be34d9ea7675ee633220777"
Using drafts with remote packages
In addition to using drafts with the buf
CLI, engineers can also consume them
with remote packages.
To reference a draft in remote packages, add a @<DRAFT_NAME>
reference to the go get
or npm install
command. The
package will be generated with the latest plugin version available when referencing the name of the draft for the remote
package.
To get a remote package with generated Go code using protocolbuffers/go for the draft shown above:
go get buf.build/gen/go/bufbuld/eliza/protocolbuffers/go@<draft-name>
To get a remote package that includes generated TypeScript code with protobuf-es:
npm install @buf/bufbuld_eliza.bufbuild_es@<draft-name>
Publishing non-breaking API changes
Do not push backwards-incompatible changes to your module.
There are exceptions to this rule for packages in development (such as
alpha
and beta
), but module authors should do everything they can to
maintain compatibility in their module.
If, for example, the Diamond Dependency Problem manifests itself, then some users may be unable to compile their module.
In the future, we plan to enable a configurable (opt-in), module compatibility guarantee so that it's impossible to push backwards-incompatible changes to your module. With this, consumers can freely update to the latest version on any module and never break their builds.
Publishing breaking API changes
If you absolutely must roll out a breaking change to your API, there are ways you can safely do so without breaking compatibility with your earlier module versions.
In the Module Layout example above, you'll notice the use of a
versioned filepath (it contains a v1
element). In this case, the filepath
reflects a versioned package that should be used in the Protobuf files in that
directory (acme.pkg.v1
).
This has two key benefits:
- The Protobuf files you define don't collide with other modules so that they can always be compiled together.
- The version element in the filepath enables you to roll out incompatible versions in the same module because they are consumed from different filepaths.
Suppose that you have a module similar to the one described in
Module Layout, and you need to make a breaking change to the
acme/pkg/v1/pkg.proto
definitions. Rather than committing a breaking change to
the same file, you can create a new file in a separately versioned filepath,
such as acme/pkg/v2/pkg.proto
.
What that looks like:
proto/
├── acme
│ └── pkg
│ ├── v1
│ │ └── pkg.proto
│ └── v2
│ └── pkg.proto
├── buf.lock
└── buf.yaml
In this case, acme/pkg/v2/pkg.proto
is incompatible with
acme/pkg/v1/pkg.proto
(the Object.id
field was changed):
syntax = "proto3";
package acme.pkg.v1;
// Object is a generic object that uses
// an int32 for its identifier.
message Object {
int32 id = 1;
}
syntax = "proto3";
package acme.pkg.v2;
// Object is a generic object that uses
// a string for its identifier.
message Object {
string id = 1;
}
Fortunately, with this structure, the module author can safely push their latest changes to the module and all of their consumers can continue to compile their modules.
The package version recommendation described here is enforceable in the
PACKAGE_VERSION_SUFFIX
lint rule which is part of the default ruleset we recommend.
Publishing commits
Every push of new content to a repository is associated with a commit that identifies that change in the schema.
The commit name is a randomly generated, fixed-size [hexadecimal] string that's visible in the BSR's UI. Note that the commit name is not a hash of the commit's content.
Commits are defined and maintained by the Buf Schema Registry and cannot be user-defined in any way. The commit is created after a successful push. This means that unlike Git, the commit only exists on the BSR repository and not locally.
To create a new commit, push your module using buf
$ buf push
Publishing tags
Tags are a user-defined reference to a single commit but with a human-readable name, similar to a Git tag. It is useful for identifying commonly referenced commits—like a release. A commit can have many tags referencing it but a tag can only reference a single commit.
$ buf push --tag <tag name>
We recommend tagging BSR commits with version control references, as a way to track corresponding revisions.
$ buf push --tag "$(git rev-parse HEAD)"