DeveloperJobs.io
← Back to all jobs

Job Description

Mercor is building production-grade Machine Learning systems for anonymization in real customer workflows. This role combines engineering ownership with high-leverage evaluation and feedback loops, so model changes translate into measurable improvements to complex work products, even when labels are noisy, delayed, or disputed.

What you’ll do

  • Build ML systems that score, validate, and improve complex work products where correctness is nuanced and labels are imperfect.
  • Design evaluation frameworks for ambiguous tasks where ground truth is partial, delayed, or disputed.
  • Create feedback loops that turn review, disagreement, correction, and adjudication into measurable improvements for both models and system behavior.
  • Own production ML behavior end-to-end, including precision/recall tradeoffs, regression detection, drift, latency, cost, and explainability.
  • Improve quality by selecting the right approach, including prompting, fine-tuning, retrieval, active learning, heuristics, and error analysis.
  • Partner with backend engineers to integrate inference into durable, long-running workflows without sacrificing debuggability or human oversight.
  • Move quickly in a young, high-ownership codebase where early design decisions carry long-term architectural weight.
  • Debug production ML failures in live, long-running workflows where silent errors matter.
  • Work across a stack that includes Python, Temporal, Postgres, AWS, and LiteLLM.
  • Balance automation confidence with human review, including knowing when to defer and when to ship.

What you bring

  • Track record of shipping ML systems that improved a real product, workflow, or business metric.
  • Strong instincts for model quality, evaluation design, error analysis, and production failure modes.
  • Comfort operating in ambiguous problem spaces where labels are imperfect and correctness evolves.
  • Sound judgment about when to use prompting, fine-tuning, heuristics, retrieval, human review, or simpler product constraints.
  • Solid engineering fundamentals across the full ML stack, not just modeling.
  • Familiarity with LLM applications, model-assisted workflows, evaluation frameworks, or human-in-the-loop ML is a strong plus.
  • 4+ years of experience.
  • A bias toward simple, inspectable ML systems that improve quickly and fail in understandable ways.
  • Ability to hold ambiguity without paralysis and make reasonable bets with incomplete information.
  • Focus on real-world outputs, not only benchmark performance.

Benefits at Mercor

  • Generous equity grant vested over 4 years
  • Health, dental, and vision insurance
  • 401(k) with company match
  • Unlimited paid vacation and sick days
  • Parental leave for birthing and non-birthing parents
  • Free mental health services
  • $200 monthly wellness reimbursement
  • $1.5K monthly meal stipend
  • $200 monthly laundry reimbursement
  • Free Equinox membership
  • Up to $15K relocation bonus

Role context: Frontier AI companies are increasingly bottlenecked on expert judgment. Mercor’s goal is to capture expert input reliably, validate it at scale, and turn it into durable model behavior. This work powers Mercor’s Frontier Data Products, where a single job can stay live for days while models, automated checks, expert review, disagreement resolution, and feedback loops interact.

Location: New York, NY (hybrid)

Compensation: USD 180,000 - 500,000 per year

Technologies

Python, Temporal, Postgres, AWS, LiteLLM

Similar Jobs