Sign in to see your match — skills, category and experience, compared honestly.Sign in
Data annotation
About the Role
Join a remote expert network as a senior rater and reviewer, evaluating challenging AI model outputs and helping calibrate other graders. Your work supports the development of reliable, high-quality AI systems.
What You'll Do
Rank and rate expert-level model answers in your domain using detailed rubrics and written rationales.
Evaluate model reasoning step by step in specialized fields such as mathematics, code, law, medicine, or finance.
Fact-check detailed claims against primary sources and verify accuracy.
Assess answers against content policies, style guides, and edge cases.
Review other raters' work, calibrate standards, and develop or refine grading guidelines.
Assess search relevance and share feedback in live sessions.
What We're Looking For
At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or deep expertise in a relevant field.
Native or near-native written English proficiency.
Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
Experience leading, reviewing, or mentoring other raters or reviewers.
An advanced degree, professional license, or deep working expertise in the domain you evaluate.
Hands-on experience with AI data annotation, preference ranking, rubric grading, process labeling, or search-quality rating is a plus.
Compensation & Benefits
Compensation is $50 to $120 per hour, depending on project, field, and expertise. Payment is for hours worked.
Location
Fully remote. Projects typically last from one week to several months.
Compensation Range: $104K - $249.6K
Originally posted on Himalayas
Verification pending— we couldn't check this link, so it's free to openJobs via Himalayas
Interview call: Priya · Hindi Transcription Specialist - Fully Remote | Up to $… at mercor
Interview call: A member · Analyst, IT Operations at Ensemble Health Partners
Interview call: Hema · Java Backend Developer at Zensar
Interview calls
🎉A member·WordPress Developer & Designer (WFH) at Augmentive Business 7 Solutions Pvt Ltd🎉A member·Customer Service Representative - Insurance at AAA🎉Omkar·AI Engineer at Zensar🎉Shaikh·Medical Records Clerk at Harris Computer🎉A member·
INTL - India - REMOTE Data Analyst/ Power BI Developer at Insight Global
🎉A member·(fluent English) Customer Support Consultant, hospitali… at SupportYourApp
🎉Nandkishor·AutoCAD Drafter (Jr. Estimator) at NightOwl Consulting
🎉Priya·Hindi Transcription Specialist - Fully Remote | Up to $… at mercor
🎉A member·Analyst, IT Operations at Ensemble Health Partners
🎉Hema·Java Backend Developer at Zensar
🎉A member·WordPress Developer & Designer (WFH) at Augmentive Business 7 Solutions Pvt Ltd
🎉A member·Customer Service Representative - Insurance at AAA
🎉Omkar·AI Engineer at Zensar
🎉Shaikh·Medical Records Clerk at Harris Computer
🎉A member·INTL - India - REMOTE Data Analyst/ Power BI Developer at Insight Global
🎉A member·(fluent English) Customer Support Consultant, hospitali… at SupportYourApp
🎉Nandkishor·AutoCAD Drafter (Jr. Estimator) at NightOwl Consulting
🎉Priya·Hindi Transcription Specialist - Fully Remote | Up to $… at mercor
🎉A member·Analyst, IT Operations at Ensemble Health Partners