StringTheory Bench — Exact Production Harness, Zero Production Impact.
Every turn runs through the exact ask-agent pipeline (live Research Forms API, BigQuery Measure Graphs, submit_answer, and sentence-level fact-checker). Vote in Blind Side-by-Side Arena to build DPO pairs, inspect the Glass-Box X-Ray to separate retrieval bugs from model reasoning, and pin hard cases to your Golden Suite for Wave C (#1867).
Pinned Golden Test Suite
Saved in BigQuery
ask_agent.dogfood_golden_suite. Replay any case against any model checkpoint.Loading...
Bench Scoreboard & Wave C Flywheel Export
Stored in BigQuery
ask_agent.dogfood_feedback (tagged with Congress scope for #1308 safety).Loading...
Access Control (Owner Only)
Grant or revoke Google accounts allowed to sign into
bench.theory.pqlabs.org. Changes take effect immediately via BigQuery (no redeploy needed).Loading...