144 jobs for "Evaluation Science Lead"

Machine Learning Engineer, Software QA - Creativity Apps

At Apple - Vancouver

Posted on Oct 19

View job Apply on company site

Cybersecurity Systems Engineer

At Leidos - Linthicum Heights

Posted on Oct 30

View job Apply on company site

Engineering Leader, AI Safety Engineering & Evaluation, Apple Services Engineering

At Apple - Cupertino

Posted on Oct 19

View job Apply on company site

AIML Sr. Privacy & Data Governance Engineer, Evaluation

At Apple - Cupertino

Posted on Oct 17

View job Apply on company site

Applied Researcher 4

At Capital One - San Jose · San Francisco

Posted on Nov 6

View job Apply on company site

Applied Researcher 4 (AI Safety & Multimodal)

At Capital One - New York · McLean

Posted on Nov 6

View job Apply on company site

Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation)

At Capital One - New York · San Francisco

Posted on Nov 5

View job Apply on company site

Senior Pharmacovigilance Scientist (Vaccines/ Products of National Significance)

At CSL

Posted on Oct 28

View job Apply on company site

Hyperspectral Image Scientist

At Leidos - Springfield

Posted on Jul 28

View job Apply on company site

Senior Product Manager, AI-Driven Content Safety - TikTok Safety Product

At TikTok - San Jose

Posted on Jun 8

View job Apply on company site

Content Risk Evaluation Team Lead (Mandarin Speaking) - Safety Model Operations - Singapore

At TikTok

Posted on Apr 28

View job Apply on company site

Content Risk Evaluation Team Lead (Mandarin Speaking) - Safety Model Operations - Kuala Lumpur

At TikTok - Kuala Lumpur

Posted on Apr 27

View job Apply on company site

Principal Data Scientist

At Aristocrat - London

Posted on Oct 9

View job Apply on company site

Agentic AI Engineer - Assistant Vice President / Vice President

At iCapital

Posted on May 27

View job Apply on company site

Associate Director, Research Biostatistics

At CSL - Schlieren

Posted on Sep 7

View job Apply on company site

Agentic AI Engineer - Assistant Vice President / Vice President

At iCapital - Jaipur

Posted on May 27

View job Apply on company site

Senior Color Developer, Global Apparel PCC (2 Years Fixed-Term Contract)

At Nike

Posted on Oct 20

View job Apply on company site

Applied Researcher 5 (AI Foundations)

At Capital One - New York · McLean

Posted on Nov 5

View job Apply on company site

Manager, Data Science - Model Risk Office

At Capital One - McLean

Posted on Nov 1

View job Apply on company site

Applied Researcher 4 (AI Foundations, VLM Customization)

At Capital One - San Jose · McLean

Posted on Nov 1

View job Apply on company site

Apple

Engineering Leader, AI Safety Engineering & Evaluation, Apple Services Engineering

Cupertino, CA$237,600 – $401,700Posted 21 days ago

Apply on company site View full job

Why you should apply for a job to Apple:

Ranked as one of the Best Technology Companies in 2020
3.7/5 in overall job satisfaction
3.1/5 in supportive management
66% say women are treated fairly and equally to men
65% would recommend this company to other women
66% say the CEO supports gender diversity

Ratings are based on anonymous reviews by Fairygodboss members. See all reviews ›

Apple Services Engineering (ASE) powers AI and machine learning experiences across the App Store, Apple Music, Apple TV, and more. We build intelligent, deeply personal experiences designed to serve users around the world. Ensuring these experiences are safe, trustworthy, and representative of the diverse communities we serve is fundamental to how we develop and deploy AI.

We are seeking an experienced engineering leader to lead AI Safety Engineering & Evaluation within the Human-Centered AI organization. This role combines software engineering, machine learning, evaluation science, and Responsible AI to develop scalable capabilities for identifying, measuring, and mitigating risks associated with AI-powered experiences.

You will lead an emerging, multidisciplinary organization of engineering managers, engineers, and applied scientists, building the technical expertise, engineering practices, and organizational capabilities needed to scale AI safety evaluation. This is an opportunity to shape a growing function, developing its technical foundations while transforming manual safety assessments into automated, scientifically rigorous processes.

The ideal candidate brings strong engineering leadership, practical expertise in machine learning and modern AI systems, and demonstrated success building technical organizations and translating complex challenges into measurable engineering outcomes.

Description

As the Engineering Leader for AI Safety Engineering & Evaluation, you will lead the development and operationalization of technical capabilities to systematically assess AI model behavior, identify potential harms, and measure the effectiveness of safety mitigations.

Your organization will advance safety evaluation methodologies, model-based assessment, adversarial testing, automated risk identification, and failure analysis. In partnership with Evaluation Systems & Research (ESR), which provides shared evaluation infrastructure and services, you will integrate safety-specific models, assessment workflows, and measurement capabilities into scalable evaluation systems.

You will establish the technical direction, organizational structure, and engineering practices needed for a growing team to operate effectively. This requires a leader who can engage deeply in technical decisions, develop engineering managers and senior contributors, and progressively build the organization's ability to deliver complex technical solutions.

You will work closely with machine learning engineering, research, product, privacy, legal, and safety policy teams to translate safety requirements into measurable technical standards and actionable insights that inform AI development and deployment decisions.","responsibilities":"Technical Strategy & Organizational Leadership: Develop and execute the technical strategy and multi-year roadmap for AI safety engineering and evaluation, aligning priorities with broader organizational objectives. Build, scale, and mentor an emerging multidisciplinary organization of engineering managers, software engineers, machine learning engineers, and applied scientists while owning staffing, resource planning, performance management, and delivery commitments. Provide hands-on technical direction across safety-specific modeling and automated assessments, establishing rigorous engineering standards, review practices, and operating mechanisms that improve execution, maintainability, and organizational effectiveness as the team matures.

AI/ML Evaluation & Safety Assessment: Lead the development of automated, model-based evaluation approaches (including LLM-as-a-judge, multi-model assessment, and automated classification) to assess models and product experiences for harmful content, bias, robustness, and Responsible AI risks. Establish scientifically rigorous evaluation methodologies, benchmarks, and experimental metrics to quantitatively measure model behavior and mitigation effectiveness. Advance synthetic data generation, scenario-based testing, and automated red teaming to identify systematic failure patterns and characterize emerging safety risks across diverse user experiences.

Engineering Execution & Scalable Automation: Direct the implementation of safety evaluation workflows, model-based assessors, automated testing logic, and risk-analysis pipelines. Partner closely with Evaluation Systems & Research (ESR) to integrate domain-specific safety capabilities into shared evaluation platforms, transitioning manual expert reviews into repeatable, automated processes embedded in model development, experimentation, and release workflows. Establish robust practices for evaluation data management, benchmark quality, test coverage, and continuous measurement, while managing technical dependencies and delivery priorities across partner engineering teams.

Cross-Functional Leadership & Influence: Partner with AI/ML engineering, research, product, privacy, legal, and safety policy teams to translate high-level safety and governance requirements into measurable technical criteria and evaluation capabilities. Influence AI development and product release decisions through evidence-based assessments of model capabilities, limitations, and mitigation efficacy. Communicate technical strategy, evaluation results, and emerging risks to senior engineering, product, and executive stakeholders, establishing cross-functional operating models that embed automated safety assessments into engineering workflows and reduce reliance on manual expert reviews.

Preferred Qualifications

Experience leading teams developing AI/ML evaluation systems, automated model assessment, or applied machine learning solutions.

Experience with generative AI and large language models, including LLM-based evaluation, agentic systems, model orchestration, or adversarial testing.

Applied experience with machine learning modeling, statistical analysis, experimental design, and quantitative evaluation methods.

Experience developing automated model assessors, synthetic data generation, red teaming, or systematic failure analysis capabilities.

Familiarity with Responsible AI principles and technical approaches to fairness, bias, robustness, harmful content, and safety risk assessment.

Experience integrating evaluation capabilities into existing engineering platforms, development workflows, or continuous measurement processes.

Experience leading multidisciplinary engineering organizations through technical transformation, particularly involving automation or applied machine learning.

Ability to translate emerging research techniques into reliable engineering implementations.

Advanced degree in Computer Science, Machine Learning, Artificial Intelligence, or a related technical discipline.

Minimum Qualifications

12+ years of experience in software engineering, machine learning engineering, applied AI, or related technical disciplines, including demonstrated experience delivering complex production-quality systems.

8+ years of engineering or applied machine learning management experience, including at least 3 years managing engineering managers or leading multiple technical teams through established managers.

Demonstrated experience building or transforming engineering organizations, including developing technical talent, establishing engineering practices, and scaling teams through periods of organizational or technical growth.

Strong software engineering foundation, including experience guiding architecture, implementation, and operationalization of complex software or AI/ML solutions.

Practical expertise in machine learning and modern AI architectures, including model development, inference, and evaluation methodologies.

Strong technical foundation in Python and modern software development practices, with the ability to assess technical tradeoffs and guide engineering execution.

Demonstrated ability to develop technical roadmaps, establish priorities, manage resources, and deliver measurable outcomes in ambiguous environments.

Excellent communication and stakeholder management skills, with experience influencing senior technical and cross-functional leaders.

Bachelor's degree in Computer Science, Machine Learning, Engineering, or a related technical field, or equivalent practical experience.

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $237,600 and $401,700, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

About Apple

Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.

Size: 10,000+ employees
Industry: Technology: Manufacturing

View company profile