408 jobs for "Senior Test Lead"

Principal Cyber Security Analyst

At Navy Federal Credit Union - Vienna

Posted on Nov 3

View job Apply on company site

Input/Output (I/O) Systems Quality Engineering Manager

At Apple - Minato-ku

Posted on Oct 19

View job Apply on company site

Senior System Administrator Team Lead

At Leidos - Alexandria

Posted on Nov 7

View job Apply on company site

Sr. Business Analyst - Operational Risk Capital

At Capital One - McLean · Richmond

Posted on Nov 2

View job Apply on company site

Senior Collections Strategy Manager

At TD Bank - Remote / Flexible

Posted on Nov 3

View job Apply on company site

Senior Signals Software Engineer - 25k Sign on Bonus

At Leidos - Columbia

Posted on Oct 30

View job Apply on company site

Custom Silicon Management Analog Design Lead - Sensors

At Apple - Austin

Posted on Oct 19

View job Apply on company site

Senior Group Manager, Audit (ATH 1146)

At TD Bank - Toronto

Posted on Oct 30

View job Apply on company site

Business Director- Strategy and Analytics (Enterprise Products, Experiences and Services Operations)

At Capital One - McLean

Posted on Oct 30

View job Apply on company site

Design Associate - Corporate Workplace

At PNC Financial Services Group - Chicago

Posted on Oct 27

View job Apply on company site

Principal Data Scientist (Credit Risk for Small Business and Commercial)

At Navy Federal Credit Union - Vienna

Posted on Oct 30

View job Apply on company site

Senior Automation Engineer (UiPath)

At Leidos - Alexandria

Posted on Oct 22

View job Apply on company site

Audit Manager II

At TD Bank - London

Posted on Oct 29

View job Apply on company site

Sr. Manager, Client Development - Card Partnerships Field Marketing

At Capital One - Chicago

Posted on Oct 29

View job Apply on company site

Design Associate - Corporate Workplace

At PNC Financial Services Group - Charlotte

Posted on Oct 27

View job Apply on company site

Director, Evaluation and Quality Assurance, Services Special Projects

At Apple - Cupertino

Posted on Oct 19

View job Apply on company site

Design Associate - Corporate Workplace

At PNC Financial Services Group - Minneapolis

Posted on Oct 27

View job Apply on company site

Senior QA Analyst (Deerfield, IL)

At Alight Solutions - Deerfield

Posted on Oct 29

View job Apply on company site

Senior Design Engineer (CPS426)

At Mitsubishi Electric Power Products - Warrendale

Posted on Nov 5

View job Apply on company site

Senior Field Service Technician – HVAC - TX

At Mitsubishi Electric Power Products - Odessa

Posted on Oct 14

View job Apply on company site

Apple

Director, Evaluation and Quality Assurance, Services Special Projects

Cupertino, CA$311,100 – $496,900Posted 21 days ago

Apply on company site View full job

Why you should apply for a job to Apple:

Ranked as one of the Best Technology Companies in 2020
3.7/5 in overall job satisfaction
3.1/5 in supportive management
66% say women are treated fairly and equally to men
65% would recommend this company to other women
66% say the CEO supports gender diversity

Ratings are based on anonymous reviews by Fairygodboss members. See all reviews ›

We are looking for a Director to build and lead the evaluation and quality assurance function for one of our most ambitious multi-year engineering initiatives. This is a true 0-to-1, first-key-hire role: There is no existing team, tooling, or process to inherit. You will define how we measure quality across an entire product end to end, build the automation and evaluation infrastructure to do it at scale (or borrow other teams' infrastructure), and build and lead the organization that

runs it.

Description

You will own evaluation and quality assurance for every layer of the product - from whether the system is being correctly triggered for the right set of user requests, through whether it surfaces and ranks the right results, to whether the underlying transactions execute correctly, including integrations with external partner APIs, and financial systems. Quality here isn't a single test suite - it spans conversational understanding, relevance and ranking, and revenue-bearing

correctness, all against a large, constantly evolving real-world dataset.

You will create and drive the evaluation and quality strategy and implementation for the product. This will extensively utilize automated grading systems and human evaluation programs where appropriate. You will make the calls about where to invest in each. You will define what "correct" and "good" mean for this product, wire that definition into continuous, always-on evaluation pipelines with clear pass/fail quality gates tied to releases, and build the team, tooling, and operating rhythm to catch issues before they reach users.

As the first hire in this space, you will operate with significant latitude and ambiguity - setting strategy, process, and technical direction with minimal existing scaffolding, while also building and scaling an organization. You will partner closely with engineering, applied ML, and program management leadership, including senior executive stakeholders, and you will be a key voice in defining what "done" and "shippable" mean for a product still taking shape, as well as iterating on decision points in production after the product ships.","responsibilities":"Design metrics and automated grading systems ("autograders") that scale to large, frequently changing datasets, utilizing those alongside human evaluation, and determining where the line between those should sit.

Wiring evaluation results into release processes as explicit quality gates, not just after-the-fact reporting.

Catching and preventing quality regressions across an entire product surface - from user-facing correctness to backend transaction integrity.

Partnering with engineering and applied ML teams to define what "quality" and "done" mean before a product's own definitions of those things have settled.

Navigating ambiguity and building structure - process, tooling, org design - in an environment where none of it exists yet.

Making judgment calls under uncertainty about where to invest limited evaluation resources for maximum risk reduction.

Preferred Qualifications

15+ years of experience in data science, machine learning evaluation, quality assurance, or test engineering roles in a technical product environment, including experience building and leading multi-level QA or evaluation organizations through managers.

Minimum Qualifications

12+ years of experience in data science, machine learning evaluation, quality assurance, or test engineering roles in a technical product environment, including experience building and leading multi-level QA or evaluation organizations through managers.

Demonstrated experience building an evaluation or QA function from the ground up - defining metrics, process, tooling, and organizational structure where none previously existed.

Extensive experience utilizing LLM-based autograders and rubrics, including rubric design, evaluation set development, validation, calibration, and human-model alignment, and using them to proactively drive product and model improvements.

Deep experience in traditional human evaluation methodology, logging, and A/B testing for consumer-facing products at scale.

Experience evaluating conversational AI, recommendation, ranking, or search systems, and - ideally - agentic or tool-use systems that take real-world actions, not just generate responses.

BS in Computer Science, Engineering, or a related technical field, or equivalent experience.

Experience with quality assurance for third-party API integrations.

Experience with logging infrastructure and instrumentation for AI product quality measurement.

Exceptional written and verbal communication skills, with a track record of presenting quality status, risk, and trade-offs clearly to, and influencing decisions among, senior executive leadership.

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $311,100 and $496,900, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

About Apple

Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.

Size: 10,000+ employees
Industry: Technology: Manufacturing

View company profile