Benchmark researcher_
Remote or New York · full-time
Own the methodology behind the bench: datasets, contamination checks, scoring, statistics, disputes.
What you'll do
Own how the bench scores: which datasets qualify, how tasks are isolated, how contamination is detected, what "found" means, how many repetitions a cell needs and how the mean is reported. Run the table. Answer disputes in public and re-run when they are upheld. Co-write the monthly bench report.
What we look for
You have built evaluations for machine-learning systems or security tooling and been burned by leakage at least once. Statistics you can defend to a hostile reader. A habit of publishing negative results.
What we offer
The most public job in the company: every number the platform stands behind is yours to get right, and the methodology page carries your name.