Sign in
Platforms
Pay
Learn AI
AI tools
Get hired
All AI training jobs

Mercor · Finance & specialist

AI Evaluation Rubric Auditor

Listed on Mercor as “AI Evaluation Rubric Auditor”

$60-$85/hrRemoteContractPaid in USD
ShareWhatsAppTelegramEmail

How this rate compares

This listing

$60–$85 per hour

Typical in Finance & specialist

$60–$90 per hour

Within the typical range

Typical: the middle of the lowest and highest rates posted by 455 listings in this category on MantaWork, in US dollars.

What this actually is

You bring your specialist expertise to AI evaluation. The shape of the work varies but the pattern is the same: review outputs, rate quality, write prompts, flag errors. The platform title (AI Evaluation Rubric Auditor) reflects the rate band and the expertise required, not the day-to-day work.

How Mercor picks people

  1. Upload your CV and answer the role's written questions.
  2. Do a video interview with Mercor's AI interviewer. Most take about 20 minutes, in English unless the role says otherwise.
  3. Some roles add a skills test. A test you pass once counts for similar roles.
  4. The project team reviews your results and sends offers to the people it picks.

Before you start

  • Take Mercor's free practice interview first. It does not count toward any application.
  • Use Chrome on a computer with a camera and microphone, in a quiet room.
  • You can retake an official interview up to 3 times, but hiring teams see how many times you retook it.

Not every applicant gets an offer. Prepare well for the interview.

How to prepare for the Mercor interview

Source: Mercor's help pages. talent.docs.mercor.com

Original posting from Mercor

The work

We are testing AI agents on realistic shopping and delivery tasks. Your role as a Project Auditor is to audit the work done by our Project Reviewers. You will audit complete tasks (prompt and grading rubric) and determine if their rubric failures are fair:

\- Was the information available?

\- Does the rubric accurately grade what the prompt asked for?

\- Would a valid answer have been marked wrong?

\- Is the rubric structure correct per the guidelines?

This role includes providing feedback to reviewers regarding prompt and rubric issues.

Must-haves

\- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.

\- Experience providing feedback to writers/reviewers

\- Consistent availability

\- Can read a long agent transcript and find where it went wrong.

\- Precise writing, and you check the data instead of trusting the reference answer.

Quoted from Mercor's public listing on 2026-11-17. We don't edit platform copy; honest framing is in the title and the “what this actually is” block above.

Mercor pays MantaWork a fee if it hires you through this link. Your pay is the same.