bench.theory.pqlabs.org

StringTheory Bench

Private post-training arena, blind A/B eval bench, and dogfood flywheel for the ST model series. Restricted to invited PQ Labs reviewers.

Checking session...

StringTheory Bench

bench.theory.pqlabs.org EXACT ask-agent + LIVE FORMS
Thread: 0 turns
StringTheory Bench — Exact Production Harness, Zero Production Impact.

Every turn runs through the exact ask-agent pipeline (live Research Forms API, BigQuery Measure Graphs, submit_answer, and sentence-level fact-checker). Vote in Blind Side-by-Side Arena to build DPO pairs, inspect the Glass-Box X-Ray to separate retrieval bugs from model reasoning, and pin hard cases to your Golden Suite for Wave C (#1867).

Pinned Golden Test Suite

Saved in BigQuery ask_agent.dogfood_golden_suite. Replay any case against any model checkpoint.
Loading...

Bench Scoreboard & Wave C Flywheel Export

Stored in BigQuery ask_agent.dogfood_feedback (tagged with Congress scope for #1308 safety).
Loading...

Access Control (Owner Only)

Grant or revoke Google accounts allowed to sign into bench.theory.pqlabs.org. Changes take effect immediately via BigQuery (no redeploy needed).
Loading...