# marmot bench (/docs/commands/bench)

<!-- Generated by scripts/gen.mjs from `marmot bench --help` and scripts/commands.mjs. Do not edit by hand. -->

Build a package's benchmarks with marmot, run them several times, and report the median and spread of each. -vs go also builds them with go test and runs both alternately, then reports the change and its significance (Mann-Whitney U, benchstat style). -save keeps the raw results; -compare FILE compares with them.

```text title="Usage"
marmot bench [package] [flags]
```

## Examples [#examples]

```sh
marmot bench -vs go ./...
marmot bench -bench Fib -count 10 .
marmot bench -save old.txt . && marmot bench -compare old.txt .
```

`marmot bench -vs go ./pkg` builds the package's benchmarks with marmot and with `go test`, runs
them alternately (`-count`, default 6), and prints the median and spread of each, the change and
whether it is significant (Mann-Whitney U, as benchstat does).

Benchmarks are built at `-O 2` unless you give a level or `-release`. A benchmark under 1 ns
per iteration gets a note: the optimizer probably removed work whose result was unused.
`-cpuprofile FILE` profiles the last run; read it with [marmot prof](/docs/commands/prof).

```sh
marmot bench -vs go -cpuprofile cpu.out ./pkg
marmot prof -tree cpu.out
```

## Flags [#flags]

Flags take one dash or two: `-release` and `--release` are the same.

| Flag | Meaning |
| --- | --- |
| `-bench string` | regexp of the benchmarks to run (default ".") |
| `-benchtime string` | time or iterations per benchmark run, as -test.benchtime |
| `-color string` | color in human output: auto, always or never (NO_COLOR turns auto off) (default "auto") |
| `-compare string` | compare with results saved by -save (or any go test -bench output) |
| `-count string` | runs of each benchmark (more runs, smaller p-values) (default "6") |
| `-cpuprofile string` | write a CPU profile of the last run to this file (with -vs go, the go build's to FILE.go); read it with marmot prof |
| `-json` | machine readable output: JSON Lines on stderr (also $MARMOT_OUTPUT=json, and the default under coding agents) |
| `-save string` | write the raw results to this file |
| `-timeout string` | time limit of each benchmark binary run (default "30m") |
| `-vs string` | go: also run the benchmarks built by go test and compare |

## Full help [#full-help]

```text title="marmot bench --help"
Build a package's benchmarks with marmot, run them several times, and report the median and spread of each. -vs go also builds them with go test and runs both alternately, then reports the change and its significance (Mann-Whitney U, benchstat style). -save keeps the raw results; -compare FILE compares with them.

Usage:
  marmot bench [package] [flags]

Examples:
marmot bench -vs go ./...
marmot bench -bench Fib -count 10 .
marmot bench -save old.txt . && marmot bench -compare old.txt .

Flags:
      --bench string        regexp of the benchmarks to run (default ".")
      --benchtime string    time or iterations per benchmark run, as -test.benchtime
      --color string        color in human output: auto, always or never (NO_COLOR turns auto off) (default "auto")
      --compare string      compare with results saved by -save (or any go test -bench output)
      --count string        runs of each benchmark (more runs, smaller p-values) (default "6")
      --cpuprofile string   write a CPU profile of the last run to this file (with -vs go, the go build's to FILE.go); read it with marmot prof
  -h, --help                help for bench
      --json                machine readable output: JSON Lines on stderr (also $MARMOT_OUTPUT=json, and the default under coding agents)
      --save string         write the raw results to this file
      --timeout string      time limit of each benchmark binary run (default "30m")
      --vs string           go: also run the benchmarks built by go test and compare
```