Home
Mercor

Applied Engineer, Evaluations Role

  • Pay: $100 to $120 / hour
  • Contract: Part time
  • Who can apply: Open worldwide
  • Posted Jun 2, 2025

In short

Earn 100 to 120 USD per hour from anywhere, on a part-time schedule. This remote mission asks you to help build and refine tests that measure how top AI coding agents perform. If you know your way around code and love digging into it, this could be a fit for you.

What you will do

  • Scout code repositories and finished pull requests that hold enough real work for tough challenges.
  • Turn those pull requests into engineering tasks, writing the prompts and the automated checks behind them.
  • Run your tests again and again across several AI models and coding tools, then spot where they break.
  • Hunt for flaws in grading, missing checks, and rules that box agents in, then write up what you find.

What you need to apply

  • Solid programming fundamentals and hands-on time in large, complex codebases.
  • Comfort with languages like TypeScript, Python, Java, Kotlin, Go, Ruby, PHP, C++ or Rust.
  • Experience writing solid tests, including edge cases, plus confidence with Git, test runners and CI tools.
  • Clear written and spoken communication, and the drive to own your work while asking questions when things are unclear.

Who can apply

Open worldwide, with no country restriction stated, so you can apply from almost anywhere.

A tip from Romowork

Our tip: point to a large project you have worked on and explain how you tested it, since that is exactly what this role is about.

Originally published on Mercor

Related jobs