Research Scientist (Remote/US/LATAM)

Research Scientist, Evaluations Anyone AI Labs — Human Data Division Reports to: CEO · Remote / LatAm / US The role You will own how Anyone AI measures frontier model capability. This is a research role at heart: you decide what a good evaluation is, design the benchmarks that prove it, and defend the methodology under lab scrutiny. You'll build frontier-grade evaluation packages across reasoning, coding, agents, tool use, and multi-modal — grounded in expert-verified truth, validated against multiple models, and QC'd to survive buyer-side review. Responsibilities - Evaluation research. Turn public benchmarks and eval targets into original evaluation designs. Own the hard questions: construct validity, discrimination, headroom, and contamination. - Benchmark development. Build evaluation packages with subject-matter experts, each with expert-verified ground truth, multi-model headroom results, and rigorous QC (calibration layers, severity-weighted rubrics, deterministic verifiers). - Experts. Recruit, calibrate, and review a pool across coding, agentic/tool-use, and STEM/reasoning. Be the final arbiter of correctness and frontier difficulty. - Lab relationships. Be a technical poin... Click Apply to read the full job description.