Home
Mercor

AI Evaluation Rubric Reviewer

  • Pay: $40 to $75 / hour
  • Contract: Contract
  • Who can apply: Open worldwide
  • Posted Nov 16, 2026

In short

Want remote pay in dollars? This freelance gig pays 40 to 75 USD per hour to help test AI agents on real-life shopping and delivery tasks. With 10 openings, it suits anyone who loves spotting what's broken and fixing it from their laptop.

What you will do

  • Review AI-drafted tasks, including the prompt and the grading rubric.
  • Go through model attempts and decide whether each failure is fair.
  • Check whether the information was available and whether the rubric grades what the prompt asked.
  • Fix unfair criteria, flag tasks that can't be saved, and write a short note on each decision.

What you need to apply

  • Solid experience writing or reviewing rubrics for LLM evaluations, with projects listed in your application.
  • Ability to read a long AI agent transcript and pinpoint where it went wrong.
  • Precise writing, plus the habit of checking the data instead of trusting the reference answer.

Who can apply

Open to applicants worldwide, with no country restriction stated. The work is remote and runs on a freelance contract.

A tip from Romowork

We'd suggest putting your rubric projects front and center in your application, since they're a must. Show a real example where you checked the data instead of trusting the reference answer.

Originally published on Mercor

Related jobs

Mercor

Scientific / ML Engineer

Pay: $90 to $140 / hour

  • Contract: Part time
  • Who can apply: United States

Posted Nov 9, 2026

Mercor

Data Analysis Expert

Pay: $70 to $120 / hour

  • Contract: Part time
  • Who can apply: Open worldwide

Posted Sep 15, 2026

Mercor

Machine Learning Expert

Pay: $70 to $120 / hour

  • Contract: Part time
  • Who can apply: Open worldwide

Posted Sep 15, 2026