Create a profile or pack
Profiles and packs are the two building blocks of a benchmark: a profile defines exactly how a benchmark runs and is scored; a pack supplies the audio, transcript, and metadata it runs against. Both are plain YAML files, validated against a JSON Schema, submitted as a pull request to the runner repository.
Contribute a profile
Point an existing runtime adapter (faster-whisper, whisper.cpp, vosk) at a model, in a new language or configuration. No audio needed — a profile is pure configuration.
Contribute a pack
Bring a dataset — an existing open corpus, or your own recordings — and package it as a benchmark pack: metadata, license, and a hash-verified manifest. Audio is never committed to the repository; you host it, the pack just points at it.
Contribute code
New runtime adapters, metrics, or hardware probes aren't YAML — they're reviewed Python code in the runner itself. See the Contribute page for that path and the review checklist every contribution goes through.