Loading opportunity…
All opportunities
Anyone AIAfterQuery 
Terac
Turing
Braintrust
micro1
Rust Labs
Verita AI
Mercor
Respondent
Expert Work · Assessment
Advanced Coding Expert
We are hiring experienced software engineers to design long-horizon coding tasks: realistic, investigation-heavy engineering problems drawn from substantial codebases, used to evaluate and train advanced AI coding agents. The ideal candidate is comfortable working across large multi-file codebases, reasoning about how
PLATFORMAfterQuery
LOCATIONRemote · eligibility varies
PAY$85/hr
SCHEDULESee role requirements
LAST CHECKED3 Nov 2026
THE OPPORTUNITY
Put your skills to work.
We are hiring experienced software engineers to design long-horizon coding tasks: realistic, investigation-heavy engineering problems drawn from substantial codebases, used to evaluate and train advanced AI coding agents. The ideal candidate is comfortable working across large multi-file codebases, reasoning about how
- Pay
- $85/hr
- Type
- Qualifying assessment for future paid work
- Work
- Remote
About the role
We are hiring experienced software engineers to design long-horizon coding tasks: realistic, investigation-heavy engineering problems drawn from substantial codebases, used to evaluate and train advanced AI coding agents. The ideal candidate is comfortable working across large multi-file codebases, reasoning about how a bug or feature ripples across modules, and building reproducible, containerized environments backed by rigorous tests. You should be able to author a problem a strong engineer would recognize as real work, and a verifier that grades a solution the way a careful reviewer would. Experience with Docker and containerized workflows is required. You will also need a substantial private codebase of your own to author tasks against.
Responsibilities
- Author long-horizon coding tasks grounded in real engineering problems (bugs, features, regressions, migrations) from substantial codebases.
- Write clear prompts that require genuine investigation and complicated execution by a coding agent.
- Build deterministic test suites and review criteria that verify correctness, catch regressions, and resist shortcut or "reward-hacking" solutions.
- Containerize tasks with Docker so they build and run reproducibly.
Required qualifications
- Proficiency in at least one modern programming language (e.g., JavaScript/TypeScript, Java, Go, Rust, C/C++, Python, etc.).
- Experience architecting or maintaining substantial, complex, and large software projects.
- Hands-on experience with Docker or similar containerization tools.
- Strong technical writing, documentation, and testing skills.
- Access to a large, complex, and private codebase you own and can author tasks against.
- Experience using coding agents for long tasks.
Preferred qualifications
- Background in code review, test authoring, or maintaining production and/or open-source systems.
- Exposure to AI evaluation, rubric/data grading, or prompt writing.
Role details are copied from the AfterQuery listing and can change. Check the platform before you apply.
Relevant skills
A few things to check
- Location and experience requirements.
- Schedule, compensation, and assessment process.
- Current availability on AfterQuery.
KEEP EXPLORING
Similar opportunities.
Related roles across other platforms, selected by field, title and skills.
Anyone AIMachine Learning Engineer – ML Evaluation & Experiment Design
Anyone AI is recruiting experienced Machine Learning Engineers for a specialized project focused on reviewing and evaluating machine learning challenges used in AI model training and evaluation.The work involves analyzing ML experiments, datasets, metrics, and pipelines to determine whether challenges are technically s
Software Engineering
Argentina · remote
$65/hr
AfterQuerySenior Domain Expert � Multi-Discipline AI Research Program
<p>We're building a research cohort of senior domain experts across seven specialist tracks � software/systems engineering, formal methods and computational science, ML inference and GPU kernels, enterprise operations, security, hardware design, and creative technology � to help evaluate how frontier AI models reason a
Remote · eligibility varies
$135/hr
Brazil & Argentina Software Engineers: Coding Tasks for AI Evaluation
We are developing a suite of coding tasks and environments to evaluate the performance of AI agents. We need experienced software developers to review and verify realistic programming tasks and evaluation harnesses.
BrazilArgentina Software EngineersRemote collaboration
Remote · eligibility varies
$40 /hr
Senior Software Engineer - AI Evaluation / Coding Agents ($100-150/hr)
Evaluate and improve AI coding models by reviewing generated code.
PythonTypescriptJavaScript
Remote · eligibility varies
$100-$150/hour, please provide a specific hourly rate expectation
BraintrustPrincipal Coding Annotator / LLM Evaluation Engineer
Leading AI Lab lists this freelance opportunity through Braintrust. Skills include Data Annotation, Software Engineering, Python.
Data AnnotationSoftware EngineeringPython
Belgium · Denmark · France +9 more
$75–$90/hr
micro1Digital Artist
Skills include image evaluation, Comparative ranking, attention to visual detail. 100 openings shown by micro1.
image evaluationComparative rankingattention to visual detail
Remote · eligibility varies
$21–$65/hr
Rust LabsProgram leads: software engineering data for AI
Leads run software engineering data programs for AI models: managing engineers and reviewers and owning quality and delivery, remotely. A one-minute form tells rustLabs about you.
Software engineeringProgram managementAI evaluation
Remote · remote
Not disclosed
Verita AIGeneralist Video Annotator
We are looking for a detail-oriented Generalist Video Annotator to support AI data annotation and evaluation projects across video, image, and text-based content. This role is ideal for someone who can carefully review videos, identify key details, and write clear, structured explanations about what they observe.Respon
Remote
$35/hr
Application Users - Adobe Illustrator on Windows - Generalist
Overview We're looking for professionals with hands on, real world experience in specialized applications. Application Expertise and Access You must already have Adobe Illustrator installed/available on your own PC running Windows , and be able to open it today.…
Arts & DesignApplication UsersAdobe Illustrator on Windows
Remote
$30–$40/hr
RespondentHow much do you know?
Two-week diary study for selected participants; apply by answering a few questions and recording a short video. Screening and researcher selection required. Incentives are subject to a 5% fee ($1 minimum).
Online · screening and location eligibility apply
$50 USD/study · before fees