Machine Learning Engineer, Evaluation - Responsible AI and Safety

Build safer, more inclusive generative AI experiences for users worldwide. Apple is seeking a Senior Machine Learning Engineer to lead Responsible AI Evaluation efforts for global generative AI products, turning policy and risk thinking into measurable evaluations that can support ship or no-ship decisions.

This onsite role is based in Cambridge, MA. The base pay range is USD 165,800 - 292,400 per year, depending on skills, qualifications, experience, and location.

What you’ll do

• Operationalize policy and risk assessments into measurable safety criteria, including designing datasets, developing LLM- and human-in-the-loop analysis methods, and implementing robust data pipelines that produce reliable, scalable insights.
• Run pre- and post-launch evaluation work, analyze results hands-on, investigate failures, and use human and automated feedback to continuously improve feature safety and user trust.
• Provide technical leadership by championing technical excellence, empowering leadership with data-driven safety recommendations, and tackling complex engineering roadblocks.
• Defend safety metrics with stakeholders and decision-makers.

Core responsibilities

• Develop metrics to evaluate safety and fairness risks in generative models and Gen-AI features.
• Design datasets, identify data needs, and scale data coverage using human and synthetic generation approaches.
• Create sampling strategies and combine human annotation with auto-grading to deliver high-confidence insights quickly and at large scale.
• Design, implement, and optimize end-to-end data pipelines, collaborating cross-functionally to execute safety evaluations.
• Increase evaluation velocity by automating manual workflows and optimizing processes to match rapid product release cycles.
• Distill complex findings into actionable insights for product engineering teams and safety policy development.
• Work with highly sensitive content, including exposure to offensive and controversial material.

Minimum requirements

• Develop metrics for evaluation of safety and fairness risks in generative models and Gen-AI features.
• Design datasets, identify data needs, and implement creative methods to scale and expand data coverage through human and synthetic generation.
• Develop sampling strategies and pair human annotation with auto-grading for high-quality, high-confidence insights at fast pace and large scale.
• Design, implement, and optimize data pipelines in collaboration with cross-functional partners for end-to-end safety evaluations.
• Automate manual workflows and optimize evaluation velocity to meet rapid product release cycles.
• Translate complex results into actionable insights for product engineering teams and safety policy development.
• Ability to work with highly sensitive content, including exposure to offensive and controversial material.

Preferred qualifications

• Experience working with multimodal models, agentic systems, and/or diverse languages and locales.
• Experience with AI compliance and regulatory needs, specifically related to child safety, clinical or health applications, or companionship chatbots.
• Experience conducting large scale benchmarking across models and capabilities.

Compensation & benefits

• Base pay range: $165,800 - $292,400 per year (depending on skills, qualifications, experience, and location).
• Opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs, including discretionary restricted stock unit awards, and ability to purchase Apple stock at a discount via the Employee Stock Purchase Plan.
• Comprehensive medical and dental coverage.
• Retirement benefits.
• A range of discounted products and free services.
• Reimbursement for certain educational expenses, including tuition, for formal education related to advancing your career.
• May be eligible for discretionary bonuses or commission payments, and relocation (as applicable).

Note: Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program. Learn more about Apple Benefits.