Lead AI Engineer

Data Solutions Org****

Hybrid

We are looking for a Lead AI Engineer to drive the development of next-generation AI and ML systems at Salesforce.

This role owns the design and evolution of intelligent decisioning systems and expands into building a broader agent flywheel (a system of self-improving feedback loops that continuously evaluate, optimize, and evolve agent performance).

This role sits on the applied side but requires strong data and systems engineering depth - you will build not just models and agents, but the data pipelines, evaluation loops, and lightweight system scaffolding that allow them to continuously improve in production.

You will build production-grade ML models, embed them into agent workflows, and define how agents learn from real-world outcomes. This is a hands-on, high-impact role focused on shipping systems that directly influence agent performance, efficiency, revenue, and customer experience.

What You'll Do

1) Build the Agent Flywheel

• Design and implement feedback loops that enable agents and ML models to self-improve over time

• Develop systems for:

  • Outcome tracking (e.g., engagement, conversions, resolution quality)
  • Agent evaluation (LLM + deterministic + human-in-the-loop signals)
  • Iterative optimization (prompting, policies, model selection, fine-tuning)

• Build pipelines that collect and structure agent traces (inputs, tool usage, intermediate steps, outputs) into high-quality training and evaluation datasets

• Close the loop from production signals → evaluation → model/prompt improvements

2) Develop Production ML & Agent Systems

• Build and deploy application-specific ML models (classification, ranking, forecasting, recommendation, etc.)

• Design and implement AI agents that combine:

  • LLM reasoning
  • Tool/API usage
  • ML-based decisioning layers

• Implement reusable agent patterns (multi-step reasoning, tool orchestration, structured outputs) within application workflows

• Integrate ML and agent capabilities into decisioning systems that drive business outcomes

3) Data & Pipeline Engineering

• Design and build scalable data pipelines (batch and near real-time) that power training, evaluation, and inference workflows

• Develop pipelines that transform raw interaction data into features, labels, and evaluation datasets

• Partner model pipelines with data pipelines to enable continuous retraining and evaluation loops

• Ensure data quality, consistency, and availability across systems

• Work with large-scale structured and unstructured data to support both ML and LLM systems

4) Evaluation, Experimentation & Optimization

• Build offline and online evaluation frameworks for agent and ML model performance

• Develop evaluation datasets, golden traces, and regression-style test sets for agent behavior

• Design and run A/B experiments to measure impact on business outcomes

• Define and monitor key metrics (quality, containment, revenue impact, latency, etc.)

• Use production traces and evaluation signals to drive continuous optimization (prompting, model selection, feature improvements, fine-tuning)

5) Architecture & Applied Systems Design

• Develop hybrid systems that blend:

  • Deterministic logic
  • Model-based scoring
  • LLM-driven generation

• Collaborate with platform teams to leverage shared infrastructure (model serving, evaluation tooling, observability), while building application-specific layers on top

• Design systems that scale with increasing agent complexity and data volume

6) Platform & API Development

• Build scalable Python services and APIs powering agent workflows

• Contribute to shared infrastructure for model serving, evaluation, and experimentation

• Ensure reliability, observability, and performance of deployed systems

Qualifications

Core Requirements

• 6+ years of experience in AI/ML engineering, applied data science, or closely related roles

• Strong hands-on experience in Python for production systems

• Proven track record building and deploying production-grade ML models

• Strong experience with data pipeline development (ETL/ELT, batch or streaming)

• Experience designing and building AI agents or agent-like systems

• Strong experience with API development and backend services

• Experience with ML lifecycle tooling (training, evaluation, deployment, monitoring)

Data & Systems Expertise

• Experience building reliable data pipelines that support ML or AI systems in production

• Familiarity with:

  • Data processing frameworks (e.g., Spark or equivalent)
  • Data orchestration tools (e.g., Airflow, Dagster, etc.)
  • Data warehousing solutions (e.g., Snowflake, BigQuery, etc.)

• Understanding of data quality, lineage, and reproducibility in ML systems

Agent & LLM Experience

• Experience building or working with LLM-powered systems (prompting, orchestration, evaluation)

• Familiarity with agent frameworks and tool-using agents

• Experience working with agent traces, evaluation datasets, or iterative improvement loops is strongly preferred

Modeling & Systems Thinking

• ****Strong understanding of:

  • Supervised learning (classification, regression, ranking)
  • Evaluation methodologies (offline + online)
  • Experimentation (A/B testing, causal inference basics)

• Ability to design systems that combine:

  • ML models

  • LLMs

  • Business logic

Engineering & Production Skills

• Experience deploying models/services in production environments

• Familiarity with:

  • Model serving architectures
  • Data pipelines
  • Monitoring and observability

• Ability to write clean, scalable, maintainable code

Preferred Qualifications

• Experience building model-driven agent improvement systems (e.g., scoring, gating, auto-optimization)

• Experience with reinforcement learning, bandits, or iterative optimization systems

• Exposure to agent evaluation tools (e.g., LangSmith, Braintrust, or similar concepts)

• Experience with large-scale experimentation platforms

• Familiarity with enterprise SaaS or CRM domains

What Success Looks Like

• Agents and production-grade ML models measurably improve over time via automated feedback loops

• Well-structured data and evaluation pipelines continuously feeding the agent flywheel

• Clear lift in key business metrics (e.g., engagement, conversion, revenue impact)

• Robust evaluation systems that enable rapid iteration and safe deployment

Unleash Your Potential

When you join Salesforce, you'll be limitless in all areas of your life. Our benefits and resources support you to find balance and be your best, and our AI agents accelerate your impact so you can do your best. Together, we'll bring the power of Agentforce to organizations of all sizes and deliver amazing experiences that customers love. Apply today to not only shape the future - but to redefine what's possible - for yourself, for AI, and the world.

Accommodations

If you need a reasonable accommodation during the application or the recruiting process, please submit a request via this Accommodations Request Form.

Please note that Salesforce uses artificial intelligence (AI) tools to help our recruiters assess and evaluate candidates' resumes and qualifications throughout the recruiting process. Humans will always make any candidate selection and hiring decisions. Please see our Candidate Privacy Statement for more information about how we use your personal data and your rights, including with regard to use of AI tools and opt out options.

Posting Statement

Salesforce is an equal opportunity employer and maintains a policy of non-discrimination with all employees and applicants for employment. What does that mean exactly? It means that at Salesforce, we believe in equality for all. And we believe we can lead the path to equality in part by creating a workplace that's inclusive, and free from discrimination. Know your rights: workplace discrimination is illegal. Any employee or potential employee will be assessed on the basis of merit, competence and qualifications - without regard to race, religion, color, national origin, sex, sexual orientation, gender expression or identity, transgender status, age, disability, veteran or marital status, political viewpoint, or other classifications protected by law. This policy applies to current and prospective employees, no matter where they are in their Salesforce employment journey. It also applies to recruiting, hiring, job assignment, compensation, promotion, benefits, training, assessment of job performance, discipline, termination, and everything in between. Recruiting, hiring, and promotion decisions at Salesforce are fair and based on merit. The same goes for compensation, benefits, promotions, transfers, reduction in workforce, recall, training, and education.