All AI training jobs

Mercor · General projects

AI Evaluation Rubric Reviewer

Listed on Mercor as “AI Evaluation Rubric Reviewer”

$40-$75/hrRemoteContractPaid in USD
ShareWhatsAppTelegramEmail

How this rate compares

This listing

$40–$75 per hour

Typical in General projects

$55–$67.5 per hour

Within the typical range

Typical: the middle of the lowest and highest rates posted by 138 listings in this category on MantaWork, in US dollars.

What this actually is

You complete short labeling tasks, record everyday activities, or evaluate AI outputs across general knowledge domains. Open to most backgrounds. The platform title (AI Evaluation Rubric Reviewer) reflects the rate band and the expertise required, not the day-to-day work.

How Mercor picks people

  1. Upload your CV and answer the role's written questions.
  2. Do a video interview with Mercor's AI interviewer. Most take about 20 minutes, in English unless the role says otherwise.
  3. Some roles add a skills test. A test you pass once counts for similar roles.
  4. The project team reviews your results and sends offers to the people it picks.

Before you start

  • Take Mercor's free practice interview first. It does not count toward any application.
  • Use Chrome on a computer with a camera and microphone, in a quiet room.
  • You can retake an official interview up to 3 times, but hiring teams see how many times you retook it.

Not every applicant gets an offer. Prepare well for the interview.

How to prepare for the Mercor interview

Source: Mercor's help pages. talent.docs.mercor.com

Original posting from Mercor

The work

We are testing AI agents on realistic shopping and delivery tasks. You'll review AI-drafted tasks (prompt, grading rubric) along with model attempts, and decide whether each failure is fair:

\- Was the information available?

\- Does the rubric accurately grade what the prompt asked for?

\- Would a valid answer have been marked wrong?

\- Is the rubric structure correct per the guidelines?

Fix the criteria that aren't fair, flag the tasks that can't be saved, and write a short note on each call.

Must-haves

\- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.

\- Can read a long agent transcript and find where it went wrong.

\- Precise writing, and you check the data instead of trusting the reference answer.

Quoted from Mercor's public listing on 2026-11-16. We don't edit platform copy; honest framing is in the title and the “what this actually is” block above.

Mercor pays MantaWork a fee if it hires you through this link. Your pay is the same.