Loading opportunity…
All opportunities
Terac
Mercor
Sovrano AI
micro1
Turing
Alignerr
Handshake AI
REX / RemoExperts
MeridialBraintrust
AI Training · freelance
Linux Experts, Model Defeating Tasks/Evaluation
Outlier lists this freelance opportunity through Braintrust. Skills include Linux.
PLATFORMBraintrust
LOCATIONUnited States · Canada · Argentina +22 more
PAY$1,350/task
SCHEDULE20 hrs/week
LAST CHECKED4 Nov 2026
THE OPPORTUNITY
Put your skills to work.
Outlier lists this freelance opportunity through Braintrust. Skills include Linux.
- Contract
- short
- Role
- Engineering
- Hours per week
- 20
- Openings left
- 100
Bring your expertise to the problems today’s most advanced AI agents still struggle to solve.
We’re looking for experienced professionals to develop challenging tasks for an AI benchmark. You’ll draw on problems from your own field and turn them into clearly defined challenges that an AI agent must solve from scratch in a Linux terminal.
What you’ll do
- Identify difficult, meaningful problems in your area of expertise.
- Translate those problems into tasks that can be solved in a Linux terminal.
- Design challenges that expose limitations in advanced AI agents. Each task is tested against three leading models, with three attempts per model, and must prove difficult enough that the models mostly fail.
Who we’re looking for
- A completed master’s degree or higher. Candidates with a bachelor’s degree, at least 10 years of relevant professional experience and a publication may also be considered.
- At least 10 years of professional experience in a technical domain.
- Expertise in coding, machine learning, systems, cybersecurity or hardware.
- At least one academic or professional publication.
- Practical comfort working in a Linux terminal.
- Currently resides in Argentina, Mexico, Puerto Rico, New Zealand, Canada, the United Kingdom, the United States, Australia, Peru, Colombia, Austria, Ireland, Spain, Luxembourg, Italy, France, Germany, Belgium, the Netherlands, Sweden, Finland, Singapore, Denmark, Norway or Switzerland. Candidates residing in New York, New Jersey or California are not eligible.
Project details
- Task-based project expected to run for at least seven weeks.
- Onboarding includes two courses and a graded assessment, taking approximately 90 minutes.
- A live onboarding webinar is also required.
Compensation
- Earn up to $1,350 per task, with opportunities for higher rates based on task quality and submission volume.
- Tasks take six to seven hours on average, although initial tasks may take longer as you get familiar with the process.
Role details are copied from the Braintrust listing and can change. Check the platform before you apply.
Relevant skills
Linux
A few things to check
- Location and experience requirements.
- Schedule, compensation, and assessment process.
- Current availability on Braintrust.
KEEP EXPLORING
Similar opportunities.
Related roles across other platforms, selected by field, title and skills.
Brazil & Argentina Software Engineers: Coding Tasks for AI Evaluation
We are developing a suite of coding tasks and environments to evaluate the performance of AI agents. We need experienced software developers to review and verify realistic programming tasks and evaluation harnesses.
BrazilArgentina Software EngineersRemote collaboration
Remote · eligibility varies
$40 /hr
Behavioral Health Expert, AI Safety and Model Evaluation
Help a leading AI lab ensure its models respond to people with care, balance and sound judgment. 1\. Overview A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations.…
Life, Physical, and Social ScienceBehavioral Health ExpertAI Safety and Model Evaluation
United States
$45–$70/hr
Sovrano AIChemical Engineer, AI Model Evaluation
Put frontier AI models through real process engineering problems and mark where their reasoning breaks. Remote, hourly, first batch about 10 hours. For chemical engineers with four or more years of hands-on industry work.
Chemical engineeringProcess designProcess safety
Remote (US, CA, UK, AU, NZ, EU)
$35–49/hr
micro1Research Engineer - Code Generation & Model Evaluation
Skills include Python3, JAVA, Rust. 100 openings shown by micro1.
Python3JAVARust
Remote · eligibility varies
$50–$100/hr
Context Elicitation & Conversation Evaluation Specialists
Review AI conversations to assess context and improve recommendations.
Business Analysis
Remote · eligibility varies
Not disclosed
Senior Software Engineer — AI Evaluation & Benchmarks
Build the benchmarks that test the world's most advanced AI models. $80–$100/hr, fully remote, Python experts wanted....
Remote
$80-100/hr
Bangla AI Evaluation Specialist
**Opportunity Overview** Handshake is looking for Bangla speakers as it’s spoken in Singapore to help make AI responses more personalized. This is not a traditional job — you'll use your language skills to help evaluate and rank AI responses so future models can better tailor their answers to individual users.…
Bangla AI Evaluation SpecialistRemote collaboration
Remote · eligibility varies
Up to $30/hr
Software Engineer – LLM Evaluation & Repository Validation
We are looking for software engineers to validate real-world repositories used to evaluate large language models. You will triage issues, set up reproducible environments, assess test quality, and check how well models fix real bugs. **Key Responsibilities** - Review and triage issues in open-source GitHub projects. -
ALL
USD $50–100 per hour, adjusted by country of residence.
JSON & Coding Evaluation Expert
Review AI-generated customer-service-style responses and technical content with a focus on coding knowledge, JSON structure, formatting accuracy and response quality.
JSONCode evaluation
Eligibility varies
$11/hr · varies by task
BraintrustClinical Pharmacist - AI Evaluation
Turing lists this freelance opportunity through Braintrust. Skills include Critical Thinking.
Critical Thinking
United States only
$38/hr