What is GOESB?
GOESB (Global Open Edge Speech Benchmark) helps developers, researchers, hardware vendors, and companies pick the optimal combination of hardware × runtime × model × configuration × dataset × language for running speech AI locally — no cloud required. It aims to be the de-facto standard for reproducible benchmarks of on-device speech processing, with first-class support for privacy-first use cases: smart home, assistive technology, robotics, healthcare, and voice assistants.
What it answers
- Which hardware runs Whisper Medium in realtime?
- Which runtime is fastest? Which model is best for a given language?
- How much power does a realtime voice assistant draw?
- Which combination gives the best price/performance? — not yet: no hardware pricing data exists (see roadmap M8).
- Which hardware is suitable for fully local AI?
Three benchmark types
Streaming is not a setting — it's a fundamental benchmark type.
| Type | Processes | Example uses | Signature metrics |
|---|---|---|---|
| Batch | whole audio at once | transcription, subtitles, minutes | WER, CER, Real-Time Factor, CPU/RAM, energy |
| Streaming | audio in realtime | voice assistants, live captions | + First Partial/Final Latency, End-of-Speech Latency, Partial Stability |
| Conversation | the full mic→VAD→ASR→LLM→TTS→speaker pipeline | complete voice assistants | Time-To-First-Response/Audio, End-to-End & Barge-in Latency |
Why trust the numbers?
Every result on this leaderboard passed a real trust gate before it was accepted.
- Hashing. The profile, model, runtime, configuration, dataset, and result are each SHA-256 hashed. Any change to any of them is detectable — a result can be traced back to the exact inputs that produced it.
- Signing. The runner requests a fresh, single-use signing token from the API right before submission (not before — producing a result never needs network access) and signs the result with it. That proves the result hasn't been altered since it was signed and which credential authorized the submission. It does not prove the underlying measurement itself is genuine — no software-only signing scheme can, without disproportionate hardware attestation infrastructure. We say this plainly rather than imply more.
- Reviewed code only. The runner never executes arbitrary code. Runtime adapters, metrics, and hardware probes are reviewed, in-tree code that ships as part of a signed runner release — adding one means opening a pull request, not injecting a script into a run.
- Official profiles, open packs. Only results run against a reviewed, versioned official profile and an open pack (identical data for everyone, so results are directly comparable) with a verified environment fingerprint appear on public leaderboards. Custom profiles and private packs exist for local use but never surface here.
Further reading: ADR-0004 and ADR-0005 for the full trust model and its honest limits, and governance.md for what keeps sponsorship and vendors from tilting a result.
Not in scope
GOESB is not a transcription service and not an AI chat platform. LLM benchmarking exists only as the reasoning-step component inside the conversation benchmark type — never a standalone LLM leaderboard. Vision models and image generation are out of scope entirely.
Ready to run a benchmark yourself? See the how-to guide, or learn how to contribute.