Skip to Content
marmot

Support matrix

Feature by feature and package by package, from the test results.

Last updated View as MarkdownAgent setup

Generated from marmot's test results of 2026-11-12. Every test builds a program with marmot and with Go and compares the output and exit status.

Status words: works (every test passes), partial (some fail), missing (not implemented), untested.

Conformance numbers

Every test compares stdout, stderr (up to the first panic line) and exit status with the stock Go toolchain.

TierPassingTotalWhat it covers
run/core134134language core, no imports, println only
run/gc1313allocation heavy programs
run/sched1515goroutines, channels, deadlock
run/std235238standard library by package
run/opt2122optimizer and specializer targets
run/mem5050use after free and lifetime hunting (every way a slice stays reachable)
run/layout4949struct layouts: every field mix, every access path
run/audit3341counterexamples from the spec and verify audits
run/apps1212200 to 600 line real programs
goroot98108single file programs from the Go project's own test suite
world2932real open source modules (vendored, pinned)
total (dev build)689714

Other build modes in the same run:

ModePassingTotal
dev689714
gcstress5050
lifetime4950

Language

FeatureStatusTests
Integers: wraparound, shifts, division, conversionsworks13 of 13
Floats: NaN, infinities, conversions, printingworks6 of 6 Fused multiply-add follows gc on arm64.
Complex numbersworks2 of 2
Strings, runes, UTF-8 iteration and conversionworks7 of 7
Slices: append, copy, aliasing, capacityworks10 of 10 Growth capacities may differ from gc.
Arrays and structs: copy, compare, embeddingworks9 of 9
Maps, all key kinds, iteration with deletionworks8 of 8 Iteration order is random, as in gc.
Methods, method values and expressionsworks2 of 2
Closures and Go 1.22 loop variablesworks4 of 4
Interfaces, assertions, type switchesworks12 of 12
Generics: functions, types, constraintsworks5 of 5
defer, panic, recoverworks18 of 18
Runtime panics: bounds, nil, divide, assertionsworks8 of 8 Message text matches gc; the stack trace format differs.
Control flow: labels, goto, switch, range over int and funcworks10 of 10
Constants and iotaworks2 of 2
Initialization order and init functionsworks2 of 2
Evaluation orderworks2 of 2 Variables are read after calls in the same expression, as gc does.
Goroutines, channels, selectworks26 of 26
Garbage collection (allocation heavy programs)works13 of 13
unsafeworks5 of 5
Compiler optimizations preserve semanticspartial21 of 22 Loop, specialization and float tests; run at -O0, -O2, -Oz and -release.
Real programs (parsers, interpreters, simulations)works12 of 12
cgomissingnon-goal for v1
Race detector (-race)missingnon-goal for v1
plugin packagemissingnon-goal for v1
DWARF debug infomissingnon-goal for v1
Windowsmissingnon-goal for v1

Standard library

Each row is a directory under tests/run/std. Packages not listed have no test yet.

PackagesStatusTests
sync/atomicworks10 of 10
bufioworks11 of 11
bytesworks10 of 10
compress/*works1 of 1
encoding/{base32,base64,hex,binary,csv,xml,gob,asn1,pem,ascii85}works4 of 4
errorsworks10 of 10
path, path/filepathworks5 of 5
fmtworks14 of 14
container/{heap,list,ring}works3 of 3
net/http, httptest, cookiejar, crypto/tlsworks8 of 8
ioworks11 of 11
encoding/jsonworks8 of 8
logpartial1 of 2
mapsworks10 of 10
math, math/bits, math/cmplxworks12 of 12
net, net/netipworks6 of 6
os, os/userworks12 of 12
math/randworks3 of 3
reflectpartial7 of 8
regexpworks6 of 6
runtime, runtime/debugmissing0 of 1
slices, cmpworks10 of 10
log/slogworks1 of 1
sortworks10 of 10
database/sqlworks1 of 1
strconvworks10 of 10
stringsworks10 of 10
syncworks12 of 12
text/templateworks5 of 5
time, time/tzdataworks14 of 14
unicode, unicode/utf8, unicode/utf16works10 of 10

Stock standard library tests under marmot test

Results of marmot test -short on stock packages (darwin/arm64 laptop, 2026-11-12, before the fixes). "alloc counts only" means every failure is a testing.AllocsPerRun count.

packageresultpassfailrate
fmtok790100%
stringsFAIL146597%
bytesFAIL175398%
strconvFAIL691186%
sortok810100%
slicesFAIL162398%
mapsok240100%
mathok1110100%
unicodeok420100%
unicode/utf8FAIL30197%
timeFAIL46296%
syncbuild00-
sync/atomicok1070100%
contextFAIL68199%
errorsok920100%
ioFAIL77297%
bufioFAIL88199%
osFAIL352511100%
os/execFAIL235729%
os/signalFAIL5183%
pathok160100%
path/filepathFAIL59198%
encoding/jsonFAIL010%
encoding/binarybuild00-
encoding/base64ok230100%
encoding/hexok180100%
encoding/csvok770100%
encoding/xmlFAIL332399%
encoding/pemok210100%
regexpok750100%
text/templateFAIL58198%
html/templateFAIL00-
text/tabwriterok60100%
text/scannerok220100%
net/urlFAIL89595%
net/httpbuild00-
net/http/httptestFAIL51394%
netFAIL90100%
mimeFAIL24196%
mime/multipartFAIL49492%
crypto/sha256FAIL33294%
crypto/md5FAIL17194%
crypto/sha1FAIL17194%
crypto/hmacok2330100%
crypto/randFAIL14288%
crypto/aesok580100%
crypto/tlsFAIL00-
crypto/x509FAIL00-
hash/crc32ok180100%
hash/fnvok620100%
container/heapok90100%
container/listok110100%
container/ringok150100%
math/randFAIL25293%
math/rand/v2ok250100%
math/bigFAIL2661100%
math/bitsbuild00-
reflectbuild00-
runtimebuild00-
logFAIL9282%
log/slogFAIL2252989%
flagFAIL28197%
embedbuild00-
io/fsFAIL50100%
compress/gzipok230100%
compress/flateok1280100%
archive/tarFAIL1233%
archive/zipFAIL00-
htmlok60100%
go/tokenFAIL25196%
go/scannerok250100%
go/parserFAIL420100%
go/astFAIL23196%
go/formatok50100%
go/typesFAIL70199%
iterFAIL101050%
uniquebuild00-

Real world modules

Vendored at a pinned version under tests/world (see tests/world/README.md), each with a small deterministic program checked against stock Go.

Module testStatus
base58works
blackfridayworks
cobraworks
decimalworks
difflibworks
doublestarworks
embedfsworks
fuzzysearchworks
gjsonworks
gocmpworks
godiffworks
goldmarkworks
gotomlworks
govaluateworks
humanizeworks
iniworks
levenshteinworks
lruworks
mapstructurefails
pflagworks
pkgerrorsworks
rateworks
semverworks
strcaseworks
tablewriterworks
testifyfails
tomlworks
ulidfails
uuidworks
xtextworks
xxhashworks
yamlworks

29 of 32 world programs pass.

Build modes

  • dev (default, -O2): the safe pipeline; every transform below that is marked off stays off.
  • -release: whole program optimization, -O3, loop and library specializations.
  • -O0 -O1 -O2 -O3 -Os -Oz: LLVM level; the conformance runner can run any program in -O0, -O2, -Oz and -release (-modes).
  • Opt-in lifetime and memory transforms, each behind an environment switch. A transform is on by default only after a proof review and a full conformance matrix plus fuzz round with it forced on; until then it is off.
TransformSwitchDefault
In-place reuse (reuse.go)MARMOT_REUSEoff
Capture by value (escape.go allocSettled)MARMOT_BYVALUEoff
Region inference (region.go)MARMOT_REGIONSoff
Frame regions (frameregion.go)MARMOT_FRAME_REGIONSoff
Allocation sinking (allocsink.go)MARMOT_ALLOCSINKoff
Escape summaries, cross package, extended rulesMARMOT_ESCAPE_SUMMARY, MARMOT_XPKG_ESCAPE, MARMOT_ESCAPE_EXToff
Conversion and make frame buffers (stackconv.go)MARMOT_STACKCONVoff
Intraprocedural escape (default path)onon
go-strings S3 no-copy (backend)-no-strings-nocopyon
Defer record elision (panics.go)MARMOT_DEFERELIDEoff
Dead initializer removal (staticinit.go)MARMOT_DEADINIToff
Allocation without zeroing (rtcore rt_alloc_nozero, 569d095; poison fill c8e0a15)callers onlyon

Stress modes used by the conformance runner: gcstress (MARMOT_GCSTRESS=1), poison (MARMOT_POISON=1), lifetime (MARMOT_LIFETIME=all).

Targets

TargetStatusTested
darwin/arm64primaryall tiers above
linux/amd64cross compiled on macOS, run on a Linux test host99 of 99 conformance tests pass (tree 2c645f1)
linux/arm64not tested

Tools

  • marmot build [flags] [packages or files]: -o, -O {0,1,2,3,s,z} (default 2), -release (whole program, -O3, loop and library specializations), -emit {mlir,llvm,obj,exe}, -dialect {llvm,go}, -gc {precise,none}, -target, -tags, -nocache, -v, -x, -work.
  • marmot run [flags] file.go [args]: build and run.
  • marmot test [-v] [-run RE] [-skip RE] [-bench RE] [-count N] [-timeout D] [packages]: builds the go list -test main and runs it in the package directory.
  • marmot version.

Known differences from gc

  • Panic output prints the same message and exits with status 2; the traceback below goroutine N [running]: is gc style but frames of inlined functions do not appear.
  • Floating point uses the same fused multiply-add rule as gc on arm64 (x*y+z fuses, an explicit conversion does not).
  • Variables are read after calls in the same expression, as gc does (the spec leaves this unspecified).
  • runtime.Caller and friends work from a pc table; binaries built with -s have none.
  • NaN payloads. Where the Go specification only says the result is a NaN, marmot may return a different NaN (payload and sign bits) than gc: the builtin min and max (also over slices, so slices.Min and slices.Max), math.Max, math.Min, and math.Floor, Ceil, Trunc, Round and RoundToEven of a signaling NaN (gc quiets it, marmot may return it unchanged). The value is a NaN in every case; only math.Float64bits, math.Float32bits or unsafe shows the difference, and gc itself differs here across architectures. Values that are not NaN are bit identical to gc on the same architecture; a difference there is a bug. Programs: tests/run/audit/minmax_nan_payload, tests/run/audit/math_exp_log_bits.
  • Slice growth capacities and map iteration order are not guaranteed to equal gc's.
  • Evaluation order: variables are read after the calls of the same expression; open cases are tracked in tests/run/audit/evalorder_gaps.
  • Unexported methods of embedded types from different packages with the same name: see world/audit_unexported under open failures.
  • cgo, the race detector, plugins, DWARF and Windows are not supported.
  • reflect: see the open failures below for the current gaps.

Open failures

Listed in tests/known_failures.txt; a test that passes is removed from the list.

  • goroot/genmeth1:
  • goroot/range4:
  • goroot/reflectmethod1:
  • goroot/reflectmethod2:
  • goroot/reflectmethod3:
  • goroot/reflectmethod4:
  • goroot/reflectmethod5:
  • goroot/reflectmethod6:
  • goroot/reflectmethod7:
  • goroot/shift3:
  • run/std/reflect/kinds_names:
  • run/opt/fma_fusion_gc: gc fuses x*y+z on arm64, marmot does not (spec allows both)
  • run/opt/panic_in_loop@Oz: recovered panic then nil deref panic is not recoverable at -Oz
  • run/std/runtime/goroutines_gc: runtime.Caller unsupported (no pc to file:line table yet)
  • run/std/log/basic: runtime.Caller unsupported, log.Lshortfile prints ???:0
  • world/ulid: return id, parse(&id): gc reads variables after calls (see core/eval_order_var_after_call)
  • run/mem/regionstore@lifetime: MARMOT_LIFETIME=all: region store use-after-free, expected until fixed
  • world/audit_unexported: unexported methods of different packages are confused in itabs
  • run/audit/deadinit_bigmake: huge make: fatal "out of memory" instead of the makeslice panic (dead initializer removal)
  • run/audit/defer_builtin_recover: recover() result lost for a deferred builtin
  • run/audit/evalorder_gaps: map index evaluation order differs from gc
  • run/audit/fma_complex_mul: complex multiply: fused multiply-add bits differ from gc
  • run/audit/make_huge_const: make with a huge constant length does not panic like gc
  • run/audit/math_exp_log_bits: math.Exp(-745.13) returns 0x1, gc 0x0 (value difference, not a NaN payload)
  • run/audit/method_value_nil_iface: method value of a nil interface panics with the wrong message
  • run/audit/tempstring_write: temporary string written through (no-copy conversion)

Support

Platforms, language, standard library, test results and known differences from Go.

FAQ

Short answers to common questions.

On this page

Conformance numbersLanguageStandard libraryStock standard library tests under marmot testReal world modulesBuild modesTargetsToolsKnown differences from gcOpen failures

Was this page helpful?

YesNo
Report an issueOpen issues