338 jobs for "Machine Learning Evaluation Engineer"

Applied Researcher 5 (AI Foundations)

At Capital One - New York · McLean

Posted on Nov 5

View job Apply on company site

Data Scientist SME

At Leidos - Englewood

Posted on Oct 16

View job Apply on company site

Internship - Machine Learning (Foundation Models)

At Apple

Posted on Oct 19

View job Apply on company site

Applied Researcher 4 (AI Foundations, VLM Customization)

At Capital One - San Jose · McLean

Posted on Nov 1

View job Apply on company site

Data Scientist SME

At Leidos - Tucson

Posted on Oct 16

View job Apply on company site

Machine Learning Engineer, Software QA - Creativity Apps

At Apple - Vancouver

Posted on Oct 19

View job Apply on company site

Manager, Data Science - Model Risk Office

At Capital One - McLean

Posted on Nov 1

View job Apply on company site

Senior Software Engineer - 3D Vision & Perception

At Apple - Sunnyvale

Posted on Oct 19

View job Apply on company site

AI Model and Agent Operation Project Intern (Business Integrity) - 2027 Start

At TikTok

Posted on Sep 17

View job Apply on company site

Data Scientist (PGC) - TikTok

At TikTok - San Jose

Posted on Aug 14

View job Apply on company site

ML Infra Engineer Intern (Ads Infra) - 2027 Summer

At TikTok - San Jose

Posted on Aug 4

View job Apply on company site

Artificial Intelligence / Machine Learning Engineer - Associate

At iCapital - New York

Posted on Oct 6

View job Apply on company site

Artificial Intelligence / Machine Learning Engineer - Associate

At iCapital - New York

Posted on Nov 6

View job Apply on company site

AIML - Machine Learning Engineer, Red Teaming - Responsible AI and Safety

At Apple - San Francisco Bay Area

Posted on Oct 19

View job Apply on company site

Applied Researcher 5

At Capital One - McLean · San Francisco

Posted on Oct 29

View job Apply on company site

ML Agent Engineer

At Apple - San Francisco Bay Area

Posted on Oct 19

View job Apply on company site

Full-stack Engineer 4 - Machine Learning Experiences (MLX)

At Capital One - San Jose · Richmond

Posted on Oct 25

View job Apply on company site

Engineering Program Manager, Apple Foundation Models, Engineering Tools

At Apple - Cupertino

Posted on Oct 19

View job Apply on company site

Applied Researcher 4

At Capital One - McLean · Cambridge

Posted on Oct 22

View job Apply on company site

Machine Learning Engineer, Apple Search & Knowledge Platforms

At Apple - Seattle

Posted on Oct 19

View job Apply on company site

Apple

AIML - Machine Learning Engineer, Red Teaming - Responsible AI and Safety

San Francisco Bay Area, CA$184,700 – $324,800Posted 20 days ago

Apply on company site View full job

Why you should apply for a job to Apple:

Ranked as one of the Best Technology Companies in 2020
3.7/5 in overall job satisfaction
3.1/5 in supportive management
66% say women are treated fairly and equally to men
65% would recommend this company to other women
66% say the CEO supports gender diversity

Ratings are based on anonymous reviews by Fairygodboss members. See all reviews ›

Would you like to play a part in protecting Apple Intelligence from real-world misuse before it reaches users? We're looking for a Machine Learning Engineer to join Apple's Responsible AI red-teaming function - a dedicated team focused on actively probing Apple Intelligence features to find real safety vulnerabilities. In this role you'll own a cluster of AI features, invent novel attacks against them, and build the tooling to scale your own techniques.

Description

This role focuses on actively red-teaming Apple Intelligence features - pre-ship and post-ship - to find real, reproducible vulnerabilities. You'll take in signals from feature risk assessments, policy definitions, evaluation results, code changes, and published attack research, and use them to probe for novel failure modes across features and models, including attacks in agentic scenarios.Producing red-teaming findings that meaningfully improve Apple's Responsible AI posture requires validating that an issue is real and reproducible, triaging its severity and reach, and driving it to resolution through the right stakeholders. This role also involves building automated red-teaming tooling - attack generation, batch orchestration, and feedback loops against known issues - to scale beyond manual testing, and partnering with a dedicated crowd and with adjacent security teams to scale efforts as well as share findings and best practices.This role heavily draws on adversarial and creative technical thinking, applied security and linguistic expertise, cross-functional communication, and sustained ownership of a feature area over time. Prior experience in AI red-teaming recommended.","responsibilities":"Own a dedicated cluster of Apple Intelligence features and build sustained expertise in their architecture and failure modesActively probe systems pre-ship and post-ship to find real, reproducible vulnerabilitiesInvent novel attack strategies and keep pace with newly published jailbreak and attack researchValidate and triage findings by severity and reach, and drive them through to mitigationDesign and build automated red-teaming tooling to scale attack generation and testing coverageIdentify gaps in policy and evaluation coverage and feed them back to eval and mitigation teamsPartner with the internal and external crowd and and with adjacent security teams to route findings appropriatelyWork with highly sensitive content, including exposure to offensive and controversial material.

Preferred Qualifications

Experience working in the Responsible AI space.Prior scientific research and publication experience.Strong organizational and operational skills working with large, multi-functional, and diverse teams.Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable.

Minimum Qualifications

MS, or PhD in Computer Science, Machine Learning, Statistics, or related fields; or an equivalent qualification acquired through other avenues.Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures.Strong engineering skills and experience in writing production-quality code in Python.Deep experience in foundation model-based AI programming (i.e.: using DSPy for optimizing foundation model prompts, for example) and a drive to innovate in this space.Experience working with noisy, crowd-based data labels and human evaluations.

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $184,700 and $324,800, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

About Apple

Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.

Size: 10,000+ employees
Industry: Technology: Manufacturing

View company profile