Loading opportunity…
All opportunities
AI Training · hourly
SWE-Bench Task Auditor
Evaluate the quality, correctness, and reproducibility of software engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric based written feedback.…
PLATFORMMercor
LOCATIONRemote — United States
PAY$70–$90/hr
SCHEDULE40 hrs/week
LAST CHECKED27 Sept 2026
THE OPPORTUNITY
Put your skills to work.
Evaluate the quality, correctness, and reproducibility of software engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric based written feedback.…
Relevant skills
Software EngineeringSWEBench Task Auditor
A few things to check
- Location and experience requirements.
- Schedule, compensation, and assessment process.
- Current availability on Mercor.