C
Senior Software Engineer, Model Evaluation
Clera
4 hours ago
Full-time
Remote
Worldwide
Remote Engineering
ABOUT THE ROLE Join an early-stage AI company building agents for manufacturing and supply chain operations. In this research-focused engineering role, you will work with procurement and manufacturing experts to create benchmarks, train models, and improve agents used in real operational workflows. WHAT YOU'LL DO - Design benchmarks and evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking. - Work with domain experts to scope tasks, write reference answers and rubrics, and build graders. - Train and fine-tune models hands-on, using evaluation results to guide experiments. - Identify agent failures, develop improvements, and validate those improvements with measured results. - Help establish research and evaluation practices as an early member of the team. WHAT WE'RE LOOKING FOR - At least 3 years of experience, with senior- or staff-level capability. - Experience designing and writing benchmarks or evaluations. - Experience authoring tasks, including defining scope, creating reference solutions or rubrics, and checking task quality. - Hands-on experience training and fine-tuning models, beyond using models through... Click Apply to read the full job description.