Reference / Inputs

Format

This page contains all the input formats, to see input specific formats see the image or source pages. Inputs have a format associated with them. Depending on the format other options are available for configuration.

Description

An option in Buf is a parameter that can be added to an input string to modify how it is interpreted by Buf commands. Inputs have a format that describes the type of the input, and each format can have specific options that apply only to that format. The most commonly used option is the format option, which can be used to override the derived format of an input. Other options include branch, tag, ref, depth, recurse_submodules, strip_components, and subdir, which are format-specific and can modify how an input is processed by Buf commands. These options are added to the input string as key-value pairs, separated by the = symbol.

The format option is used to specify the format of schemas in the format of images and sources, as well as the format of payloads when using commands like buf convert which has the flags --to and --from which can use format options.

Usage

Options are parsed to input parameters in the following form:

path#option_key1=option_value1,option_key2=option_value2

Options

#format=[dir|git|mod|protofile|tar|zip|bin|json]

Format	Description	Derived extension	Example
dir	directory		`path/to/project`
git	git repository	.git	`https://github.com/foo/bar.git`
mod	bsr module		`buf.build/bufbuild/eliza`
protofile	protobuf file	.proto	`path/to/file.proto`, `path/to/file#format=protofile`
tar	tarball	.tar	`archive.tar`, `archive#format=tar`
zip	zip archive	.zip	`archive.zip`, `archive#format=zip`
bin	protobuf binary	.bin	`image.bin`, `image#format=bin`
json	json image	.json	`image.json`, `image#format=json`

#format=dir

A local directory. The path can be either relative or absolute. This is the default format. By default, buf uses the current directory as its input for all commands.

Usage

#format=dir

#format=mod

A Module on the Buf Schema Registry. This uses whatever is in this module for the sources.

Usage

#format=mod

#format=git

A Git repository. The path to the Git repository can be either a local .git directory, or a remote http://, https://, ssh://, or git:// location.

When format is git the ref, branch, tag, depth, recurse_submodules, and subdir options are available.

Usage

#format=git,ref=<ref>,branch=<branch>,tag=<tag>,depth=<depth>,recurse_submodules=<bool>,subdir=<subdir>

Option	Description	Default	Example
ref	A valid git ref	HEAD	`buf build https://github.com/foo/bar#format=git,ref=my_branch`
branch	A valid git branch	HEAD	`buf build path/to/module#format=git,branch=my_branch`
tag	A valid git tag		`buf build https://github.com/foo/bar#format=git,tag=v1.0.0`
depth	A valid git depth	50	`buf build path/to/module#format=git,depth=1`
recurse_submodules	Whether to clone submodules recursively	false	`buf build https://github.com/foo/bar#format=git,recurse_submodules=true`
subdir	Which subdir to use as base directory		`buf build path/to/archive.tar#format=git,subdir=bar`

#format=tar

A tarball. The path to this tarball can be either a local file, a remote http/https location, or stdin. To use stdin use -, for example cat archive.tar | buf build -#format=tar.

When format is tar the compression, strip_components, and subdir options are available.

Usage

#format=tar,compression=[gzip|zstd],subdir=<subdir>,strip_components=<integer>

Options

Option	Value	Description	Example
compression	gzip,zstd	compression to use	`buf build image#format=tar,compression=gzip`
strip_components	integer	strip first n directories, applied before subdir	`buf build path/to/archive.tar#format=tar,strip_components=2`
subdir	string	subdirectory to use as base directory	`buf build path/to/archive.tar#format=tar,subdir=bar`

Derived extensions

As well as using the format and compression options, the following file extensions can be used instead.

Extension	Derived Compression	Example
.tar	none	archive.tar
.tar.gz	gzip	archive.tar.gz
.tgz	gzip	archive.tgz
.tar.zst	zstd	archive.tar.zst

#format=protofile

A local .proto file. The path can be either relative or absolute, similar to the dir input. This is a special input that uses the file and its imports as the input to buf commands. If a local configuration file is found, dependencies specified are used to resolve file imports first, followed by the local filesystem. If there is no local configuration, the local filesystem is used to resolve file imports.

When format is protofile The include_package_files option is available.

Usage

#format=protofile,include_package_files=<bool>

Option	Value	Description	Example
include_package_files	bool	include all other files in the package for the specified `.proto` file	`buf build file.proto#format=protofile,include_package_files=true`

#format=zip

A zip archive. The path to this archive can be either a local file, a remote http/https location, or stdin. To use stdin use -, for example cat archive.zip | buf build -#format=zip.

When format is zip the compression, subdir, and strip_components options are available.

Usage

#format=zip,compression=[gzip|zstd],subdir=<subdir>,strip_components=<integer>

Options

Option	Value	Description	Example
compression	gzip,zstd	compression to use	`buf build image#format=zip,compression=gzip`
strip_components	integer	strip first n directories, applied before subdir	`buf build path/to/archive.tar#format=zip,strip_components=2`
subdir	string	subdirectory to use as base directory	`buf build path/to/archive.tar#format=zip,subdir=bar`

#format=bin

Protobuf binary format. The path to this archive can be either a local file, a remote http/https location, or stdin. To use stdin use -, for example cat image.bin | buf build -#format=bin or just cat image.bin | buf build -.

When the format is bin the compression option is available.

Usage

#format=bin,compression=[gzip|zstd]

Options

Option	Value	Description	Example
compression	gzip,zstd	compression to use	`buf build image#format=bin,compression=gzip`

Automatically derived formats and compressions

As well as using the format and compression options, the following file extensions can be used instead.

Extension	Derived Compression	Example
.bin	none	image.bin
.bin.gz	gzip	image.bin.gz
.bin.zst	zstd	image.bin.zst

#format=json

Protobuf in JSON format. This creates files that use much more space and are slower to parse but result in diffs that show the actual differences between two Protobuf objects in a readable format. The path to this archive can be either a local file, a remote http/https location, or stdin. To use stdin use -, for example cat image.json | buf build -#format=bin.

When the format is json, the compression option is available.

Usage

#format=json,compression=[gzip|zstd]

Options

Option	Value	Description	Example
compression	gzip,zstd	compression to use	`buf build image#format=json,compression=gzip`

Automatically derived formats and compressions

As well as using the format and compression options, the following file extensions can be used instead.

Extension	Derived Compression	Example
.json	none	`image.json`
.json.gz	gzip	`image.json.gz`
.json.zst	zstd	`image.json.zst`

Automatically derived formats and compressions

By default, buf derives the format and compression of an input from the path via the file extension.

Extension	Derived format	Derived Compression
.tar	tar	none
.tar.gz	tar	gzip
.tgz	tar	gzip
.tar.zst	tar	zstd
.zip	zip	none
.git	git	none
.proto	protofile	none
.bin	bin	none
.bin.gz	bin	gzip
.bin.zst	bin	zstd
.json	json	none
.json.gz	json	gzip
.json.zst	json	zstd

There are also two special cases:

If the path is -, this is interpreted to mean stdin. By default, this is interpreted as the bin format.

Of note, the special value - can also be used as a value to output parameters, which is interpreted to mean stdout, and also interpreted by default as the bin format.
If the path is /dev/null on Linux or Mac, or nul for Windows, this is interpreted as the bin format.

If no format can be automatically derived, the dir format is assumed, meaning that buf assumes that the path is a path to a local directory.

The format of an input can be explicitly set as described above.

Deprecated formats

The formats below are deprecated. Though they may continue to work, we recommend updating if you are explicitly specifying any of these.

Format	Replacement
`bingz`	Use the `bin` format with the `compression=gzip` option.
`jsongz`	Use the `json` format with the `compression=gzip` option.
`targz`	Use the `tar` format with the `compression=gzip` option.

Examples

Build an image in tar format with gzip compression:

buf build path/to/image#format=tar,compression=gzip

Invoke an RPC using a git repo as the input to buf curl --schema:

buf curl --data '{"sentence": "I feel happy."}' https://demo.connect.build/buf.connect.demo.eliza.v1.ElizaService/Say --schema=https://github.com/bufbuild/connect-demo.git

Invoke an RPC using a binary as the input to buf curl --schema:

buf curl --data '{"sentence": "I feel happy."}' https://demo.connect.build/buf.connect.demo.eliza.v1.ElizaService/Say --schema=eliza.bin

Lint a dir of protobuf files:

buf lint path/to/directory#format=dir

Export from a tar archive with gzip compression:

buf export path/to/archive#format=tar,compression=gzip --output=path/to/output

Export a git repository:

buf export https://github.com/example/repo#format=git --output=path/to/output

Use a zip as the source for buf convert:

buf convert path/to/archive.zip#format=zip,compression=zstd --from=file.json --to=file.bin --type Person

Lint a git repository on a specific branch:

buf lint https://github.com/example/repo#format=git,branch=my_branch