Turing
Job Description
Model Evaluator Javascript
Full-time
Remote
SkillsJavaScript
Overview
About Turing
Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
About the Role:
We are seeking experienced Software Engineers to support a frontier AI training and evaluation initiative focused on improving the coding capabilities of AI agents.
In this role, you will review and compare AI-generated coding solutions across real-world software engineering tasks. You will analyze complete agent execution workflows, evaluate code correctness, efficiency, safety, and overall quality, and provide structured ratings and evidence-based feedback to help improve AI-assisted software development
What you’ll do
You’ll review three AI coding agents that were each given the same real developer request.
You read each agent’s full run: the commands it ran, the files it changed, what it checked and its final message.
Then you judge which run did the job best.
For every task you
Rate each run on 10 quality areas, such as following instructions, whether the code works, efficiency, safety, honesty and communication.
Give each run an overall score from 1 to 5 and rank the three.
Write a short rationale explaining your ranking, with evidence from the runs.
Required Skills:
Minimum 3+ years of overall experience
Strong experience in JavaScript / TypeScript
Proficiency with Git, Docker, and basic software pipeline setup.
Ability to understand and navigate complex codebases.
Comfortable running, modifying, and testing real-world projects locally.
Assessment & Selection Process
Candidates must complete three practice evaluation tasks as part of the selection process.
Each assessment requires quality ratings, overall scores, comparative rankings, and written rationales.
A reviewer will assess each task for technical accuracy, consistency, and quality of reasoning.
Candidates will be onboarded only after all three practice tasks have been approved.
IMPORTANT — STRICT NO-AI POLICY
DO NOT USE CHATGPT OR ANY OTHER AI TOOL TO WRITE YOUR EVALUATION RATIONALES, EXPLANATIONS, OR REASONS.
ALL RATIONALES AND REASONING MUST BE WRITTEN INDEPENDENTLY BY THE CANDIDATE.
COMPLIANCE WILL BE CHECKED. ANY USE OF AI-GENERATED RATIONALES OR REASONS WILL RESULT IN AUTOMATIC REJECTION.
Perks of Freelancing With Turing
Work in a fully remote environment.
Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details
Commitments Required: At least 8 hours per day and 40 hours per week with overlap of 4 hours with PST.
Employment type : Contractor assignment (no medical/paid leave)
Duration of contrac : 1 month; [expected start date is next week]
Payment model: $25 per approved task
Location: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico, LA
Evaluation Process (approximately 75 mins)
Assessment based review
Source: Company Career page
To apply for this job please visit work.turing.com.