Algorithm Evaluation Manager
22 days ago Sunnyvale, CA
About us
Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.
- Size: 10,000+ employees
- Industry: Technology: Manufacturing
Why you should apply for a job to Apple:
Ranked as one of the Best Technology Companies in 2020
3.7/5 in overall job satisfaction
3.1/5 in supportive management
66% say women are treated fairly and equally to men
65% would recommend this company to other women
66% say the CEO supports gender diversity
Ratings are based on anonymous reviews by Fairygodboss members. See all reviews ›
We are looking for an experienced and highly motivated Engineering Manager to lead a dynamic team focused on machine learning algorithms or Large Language Model (LM) evaluation. In this role, you will guide a team responsible for the critical data and evaluation pipelines that ensure our models are accurate, robust, and performant. The ideal candidate will bring a strong mix of technical leadership, expertise in data curation and annotation processes, and deep analytical skills. You will collaborate closely with cross-functional research and engineering teams, requiring exceptional communication and strategic thinking.
Description
As the Engineering Manager for this team, you will be at the forefront of our AI/ML development lifecycle. Your day-to-day responsibilities will include:
- Team Leadership: Lead, mentor, and grow a team of engineers and data specialists. Foster a culture of innovation, rigorous analysis, and continuous learning.
- Evaluation Strategy: Define and execute the evaluation strategy for both CV and LM models. Build robust, scalable evaluation pipelines that accurately reflect real-world performance.
- Data Pipeline Management: Oversee the end-to-end data lifecycle. This includes establishing data curation guidelines, managing data quality, and optimizing large-scale annotation workflows with external vendors or internal teams.
- Analytical Deep Dives: Guide the team in performing rigorous data analysis to troubleshoot model regressions, uncover data quality issues, and identify opportunities for algorithmic improvements.
- Strategic Alignment: Act as the primary point of contact for your team, communicating progress, bottlenecks, and strategic data needs to leadership and partner teams.
Preferred Qualifications
Advanced Degree: PhD in Computer Science, Machine Learning, or a related field.
Deep Domain Knowledge: Expertise in both Computer Vision (CV) algorithms and Large Language Model (LM) evaluation methodologies (e.g., RLHF, prompt evaluation).
Scale & Operations: Experience scaling large data operations, managing complex annotation workflows, and working directly with external data vendors.
Technical Stack: Familiarity with Python, SQL, and ML frameworks (e.g., PyTorch) to effectively review technical work and guide engineering decisions.
Cross-Functional Leadership: Demonstrated ability to drive strategic alignment across downstream product teams, ML researchers, and platform engineers in a highly matrixed environment.
Minimum Qualifications
Education & Experience: BS and a minimum of 10 years relevant industry experience
Management Experience: 2+ years of direct people management experience, with a track record of hiring, mentoring, and leading high-performing technical teams.
Domain Expertise: Proven experience in model evaluation, benchmarking, and A/B testing methodologies for machine learning models (Computer Vision or Foundation Models).
Inference Infrastructure: Familiarity with the design and architecture of machine learning inference pipelines and underlying infrastructure.
Data & Annotation: Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation processes.
Data Analysis: Strong analytical skills with the ability to dive deep into datasets to identify trends, biases, and areas for model improvement.
Communication: Excellent verbal and written communication skills, with the ability to translate complex technical concepts to both technical and non-technical stakeholders.
Pay & Benefits
At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $206,200 and $356,400, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits
Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.
Description
As the Engineering Manager for this team, you will be at the forefront of our AI/ML development lifecycle. Your day-to-day responsibilities will include:
- Team Leadership: Lead, mentor, and grow a team of engineers and data specialists. Foster a culture of innovation, rigorous analysis, and continuous learning.
- Evaluation Strategy: Define and execute the evaluation strategy for both CV and LM models. Build robust, scalable evaluation pipelines that accurately reflect real-world performance.
- Data Pipeline Management: Oversee the end-to-end data lifecycle. This includes establishing data curation guidelines, managing data quality, and optimizing large-scale annotation workflows with external vendors or internal teams.
- Analytical Deep Dives: Guide the team in performing rigorous data analysis to troubleshoot model regressions, uncover data quality issues, and identify opportunities for algorithmic improvements.
- Strategic Alignment: Act as the primary point of contact for your team, communicating progress, bottlenecks, and strategic data needs to leadership and partner teams.
Preferred Qualifications
Advanced Degree: PhD in Computer Science, Machine Learning, or a related field.
Deep Domain Knowledge: Expertise in both Computer Vision (CV) algorithms and Large Language Model (LM) evaluation methodologies (e.g., RLHF, prompt evaluation).
Scale & Operations: Experience scaling large data operations, managing complex annotation workflows, and working directly with external data vendors.
Technical Stack: Familiarity with Python, SQL, and ML frameworks (e.g., PyTorch) to effectively review technical work and guide engineering decisions.
Cross-Functional Leadership: Demonstrated ability to drive strategic alignment across downstream product teams, ML researchers, and platform engineers in a highly matrixed environment.
Minimum Qualifications
Education & Experience: BS and a minimum of 10 years relevant industry experience
Management Experience: 2+ years of direct people management experience, with a track record of hiring, mentoring, and leading high-performing technical teams.
Domain Expertise: Proven experience in model evaluation, benchmarking, and A/B testing methodologies for machine learning models (Computer Vision or Foundation Models).
Inference Infrastructure: Familiarity with the design and architecture of machine learning inference pipelines and underlying infrastructure.
Data & Annotation: Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation processes.
Data Analysis: Strong analytical skills with the ability to dive deep into datasets to identify trends, biases, and areas for model improvement.
Communication: Excellent verbal and written communication skills, with the ability to translate complex technical concepts to both technical and non-technical stakeholders.
Pay & Benefits
At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $206,200 and $356,400, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits
Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.
Additional Information
- Salary: $206,200 – $356,400
- Posted: 22 days ago