Build a profile

Fill in the fields below; the YAML preview on the right updates live. Everything stays in your browser — nothing is uploaded. Download the file, add it under profiles/<your-id>/profile.yaml in a clone of the runner repo, and open a pull request — see the field reference for what each field means and the exact submission steps.

A profile is pure configuration for a runtime that already exists — no installing, no dependencies, nothing to build. It says which existing runtime + model to run and how to score it, nothing more.

idRequired.version

title (optional)

benchmark_typelanguage (optional)BCP-47, e.g. de-DE

Runtime

A profile can only point at a runtime that already exists — picking one here doesn't install anything or need any setup. Want an engine that isn't listed? That's a code contribution, not a form.

nameRequired.min_version (optional)

Model

nameRequired.quantization (optional)beam_size (optional)temperature (optional)

vad

Configuration (optional)

threadschunk_ms

Normalization

lowercaseremove_punctuationexpand_numbers

ruleset_id (optional)e.g. goesb-en-v1, goesb-nl-v1 — a new language usually needs a new ruleset

Scoring & metricsprimary_metricRequired.tie_breakers (optional, comma-separated)metrics (comma-separated)At least one metric id, comma-separated.

changelog notes (optional)

profile.yaml preview

Fill in every required field (marked in red above) to enable the download.

version: 1.0.0
benchmark_type: batch
runtime:
model:
normalization:
  lowercase: true
  remove_punctuation: true
  expand_numbers: true
scoring:
metrics: []
changelog:
  - version: 1.0.0
    notes: Initial profile.