Model quality benchmark

Compare connected approved models on the same versioned tasks: credit conversion, queue-state recovery, local-path validation, booking conflicts, and account isolation. Each run starts in a separate benchmark project. Customer projects are never modified.

No model rankings are published yet. Automatic selection currently uses task difficulty and your own recent provider completion, credit, and latency measurements. A completed response does not prove the generated code works.

Repeatable isolated checks

The runner measures JavaScript module builds, fixed behavioral tests, up to one repair, elapsed time, and actual Kigo credits per successful result. Failed attempts count toward cost. Unknown usage stays unknown. These small fixtures do not certify React apps, mobile layouts, or production readiness.

1. Download all three files into one folder: run.mjs, suite.mjs, and check.mjs.
2. Use Node 22+ and Docker with an already installed Node image pinned by digest. Set KIGO_BENCH_IMAGE to that image. Generated code runs without network access, credentials, or writable host files.
3. Inspect the free dry run with node run.mjs.
4. Choose your own total and per-request credit ceiling, then run node run.mjs --run --max-credits 10 --per-request 2. These example limits are ceilings, not a price estimate. Approve the displayed device sign-in code in your Kigo account.
5. Results save to kigo-benchmark-results.json after each task. The script stops when the budget cannot cover another request. It uses existing Kigo credits and spending limits; it never calls provider APIs with separate keys.

Keep the same suite version, pinned image, connected models, and limits when comparing runs. Repeat the suite before drawing conclusions from a small sample. Incomplete coverage and unknown costs should not be presented as a winning score.

Benchmark projects remain in your workspace for inspection. You can delete them afterward. Revoke the benchmark device session from your account when finished.