Loading opportunity…
All opportunities
Mercor 
Terac
Braintrust
Handshake AI
Anyone AI
Meridial
Verita AI
micro1
Sovrano AI
Turing
AI Training · hourly
AI Developer Trace Task Auditor
Evaluate the quality and correctness of AI assisted software development traces used to train and evaluate a frontier AI lab's models. You'll assess end to end coding sessions produced with AI assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric based written…
PLATFORMMercor
LOCATIONRemote — United States
PAY$70–$90/hr
SCHEDULE40 hrs/week
LAST CHECKED11 Nov 2026
THE OPPORTUNITY
Put your skills to work.
Evaluate the quality and correctness of AI assisted software development traces used to train and evaluate a frontier AI lab's models. You'll assess end to end coding sessions produced with AI assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric based written…
- Commitment
- hourly
- Hours per week
- 40
- Field
- Software Engineering
- Open slots
- 3
- Eligible countries
- USA
Evaluate the quality and correctness of AI-assisted software-development traces used to train and evaluate a frontier AI lab's models. You'll assess end-to-end coding sessions produced with AI-assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric-based written feedback.
Basic Qualifications
Preferred Qualifications
- 3+ years professional software development
- Hands-on experience with AI-assisted coding tools and agentic / spec-driven workflows (Cursor, GitHub Copilot, Claude Code, or similar)
- Strong code-reading and debugging skills across full-stack or backend systems
- Ability to evaluate multi-step coding trajectories for correctness and best practice
- Experience with Kiro or Amazon CodeCatalyst
- Prior work evaluating or grading AI-generated code
- Contributions to developer tooling
Role details are copied from the Mercor listing and can change. Check the platform before you apply.
Relevant skills
Software EngineeringAI Developer Trace Task AuditorRemote collaboration
A few things to check
- Location and experience requirements.
- Schedule, compensation, and assessment process.
- Current availability on Mercor.
KEEP EXPLORING
Similar opportunities.
Related roles across other platforms, selected by field, title and skills.
Kubernetes Task Auditor
Evaluate the quality, correctness, and production readiness of Kubernetes tasks used to train and evaluate a frontier AI lab's models. You'll assess cluster operations scenarios, manifest correctness, and failure mode troubleshooting — and provide clear, rubric based written feedback.…
Software EngineeringKubernetes Task AuditorRemote collaboration
Remote — United States
$70–$90/hr
Finance and Accounting Professionals: Business Operations and Software Workflows
We are running a paid interview study on the software tools used for everyday finance and accounting workflows. We need business operations professionals to walk us through their daily processes and evaluate their current tech stack.
Finance and Accounting ProfessionalsBusiness Operations and Software WorkflowsRemote collaboration
United States · remote
$71 one-time
BraintrustLead Security Engineer (US-Remote)
Handl Health lists this direct_hire opportunity through Braintrust. Skills include Security Engineering, Cloud Security, Security Auditing.
Security EngineeringCloud SecuritySecurity Auditing
United States only
$185,000–$215,000/yr
Energy Auditor
Handshake is looking for building energy and auditing professionals to support AI research through flexible, hourly contract work. This is not a traditional job. You'll draw on your hands-on experience with building energy modeling, energy auditing, and performance benchmarking to evaluate AI-generated content and…
Energy AuditorRemote collaboration
Remote · eligibility varies
Up to $80/hr
Anyone AIFP&A Analyst
About the opportunityAnyone AI is seeking FP&A Analysts to develop realistic tasks that test AI agents’ ability to analyze business performance and support financial planning. You will create original tasks and reference models, then evaluate agents working with spreadsheets, planning software, and financial datasets.C
AI Training
Fully Remote · remote
$80–$150/hr
Technical Task Auditor
Audit technical tasks for accuracy, realism, reproducibility and tests. Specialties include GPU kernels, security, machine learning, AWS, Kubernetes and SWE-Bench.
Eligibility varies
$60/hr · varies by task
Verita AITask & Evaluation Specialist
What this role does Design realistic, difficult shopping-related tasks (e.g. meal planning, restocking, gifting, travel prep) that stress-test AI models, then build the rubrics used to grade how well a model performs them. Who we're looking for Subject-matter expertise in retail, e-commerce, consumer shopping behavior,
Software Engineering
Remote · eligibility varies
$60/hr
micro1AI Trainer (SWE)
Skills include Software Engineering, AI training. 100 openings shown by micro1.
Software EngineeringAI training
Remote · eligibility varies
$60–$120/hr
Sovrano AISoftware Engineering Expert, AI Trainer
Fully remote, project-based work checking how well AI coding agents handle real software engineering. You write tasks based on your own experience and grade where the agent gets it wrong. $40 to $125 USD an hour, at least 10 hours a week, as an independent contractor. For working software engineers in the US, UK, Irela
Software engineeringCode reviewDebugging
Remote (US, CA, UK, AU, IE)
$40–125/hr
Remote Full Stack Developer
Create and refine full-stack solutions for AI model training and evaluation.
PythonJavaScriptJava
Remote · eligibility varies
Not disclosed