Apple is seeking a Machine Learning Engineer to lead evaluation and quality for ML models and AI features within its Creativity Apps. Based in San Diego, CA in an onsite role, you will shape how quality is measured and improved, ensuring creators get reliable, high-performing experiences as models evolve from development through shipping.
Role Summary
You will define evaluation strategy and metrics for ML models, LLMs, and AI features, then build the datasets and infrastructure needed to run scalable, reproducible quality assessments. Working across ML engineering, infrastructure, product, and design, you will translate real creator experiences into practical testing methodology and clear quality reporting for technical and non-technical stakeholders.
Responsibilities
- Own evaluation and quality of AI features across Apple Creativity Apps.
- Partner with Machine Learning engineering, infrastructure, product, and design teams to convert models and use cases into evaluation methodology, testing strategy, and quality metrics.
- Define evaluation strategy and metrics tailored to specific models, LLMs, product behavior, and use cases.
- Curate and create evaluation datasets, measure and improve dataset quality, and run iterative loops to keep datasets representative.
- Design, build, maintain, and optimize evaluation infrastructure at scale with reproducible runs, tracked baselines, and model version comparisons.
- Use ML, LLMs, and agentic workflows to improve evaluation through automated judging, test generation, and analysis that scales.
- Drive end-to-end evaluation and quality from model checkpoints to integrated features shipped to customers, including triaging issues and pushing them to resolution.
- Report quality in a way that translates results into ship/no-ship risk and recommendations for technical and non-technical audiences.
- Work through ambiguity by partnering cross-functionally and making educated, critical judgments.
Requirements
- 5+ years of industry experience in machine learning or software engineering, focused on ML/AI evaluation and quality.
- Experience building evaluation methodology for ML models, LLMs, and AI products or features.
- Strong understanding of the ML development cycle, including data, training, evaluation, deployment, and monitoring.
- Solid ML fundamentals and LLM knowledge, including agentic workflows, with an ability to keep up with new developments.
- Track record applying LLMs and/or agentic workflows to ML evaluation and quality.
- Experience with data curation, dataset creation, and dataset quality using iterative improvement workflows.
- Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale.
- Experience designing evaluation metrics based on the model, product, and use cases.
- Experience driving end-to-end evaluation and quality for AI products.
- Fluency in quality reporting and communicating quality data to technical and non-technical audiences.
- Strong communication skills and experience driving cross-functional partners.
- Proficiency in Python and/or Swift or comparable languages.
- Bachelor’s degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.
Technologies
Benefits
- Comprehensive medical and dental coverage.
- Retirement benefits.
- Discounted products and free services.
- Reimbursement for certain educational expenses (including tuition) for formal education related to advancing your career.
- Opportunity to become an Apple shareholder through Apple’s discretionary employee stock programs.
- Discretionary restricted stock unit awards.
- Ability to purchase Apple stock at a discount via the Employee Stock Purchase Plan.
- Discretionary bonuses or commission payments, and possible relocation eligibility.
- Base pay range: USD 142,300 - 263,300 per year, determined based on skills, qualifications, experience, and location.
Preferred Qualifications
- Knowledge of the software development life cycle and software quality assurance methodologies.
- Experience in objective image and video quality evaluation and metrics.
- Knowledge of color science and/or experience with subjective image or video quality review.
- Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus.
- Excitement about learning new technology and applying it to practical problems.
- Familiarity with iOS and macOS.
Location: San Diego, CA (onsite)
Salary: USD 142,300 - 263,300 per year
