6 jobs for "Evaluation Reliability SRE"

LLM / Dataset Operations Project Intern (TikTok Live) - 2027 Start

At TikTok

Posted on Nov 9

View job Apply on company site

Mortgage Loan Officer- Chatham NJ

At TD Bank - Chatham

Posted on Nov 9

View job Apply on company site

Senior Sourcing Manager

At Chewy - Plantation

Posted on Nov 9

View job Apply on company site

AI Agent & Automation Project Intern (TikTok Live) - 2027 Start

At TikTok

Posted on Nov 9

View job Apply on company site

TikTok Shop - Customer Experience & Aftersales Operations Manager

At TikTok

Posted on Nov 9

View job Apply on company site

EMEA Partner Operations Manager - Graduate (TikTok AI Data Service and Operations) - 2027 Start

At TikTok - Dublin

Posted on Nov 9

View job Apply on company site

TikTok

LLM / Dataset Operations Project Intern (TikTok Live) - 2027 Start

SingaporeOperationsPosted today

Apply on company site View full job

Why you should apply for a job to TikTok:

4.5/5 in overall job satisfaction
4.5/5 in supportive management
100% say women are treated fairly and equally to men
100% would recommend this company to other women
100% say the CEO supports gender diversity

Ratings are based on anonymous reviews by Fairygodboss members. See all reviews ›

Responsibilities

About the AIDO Team
AIDO supports TikTok LIVE governance through market operations, quality management, and AI-enabled evaluation. Our Market and LLM teams are moving toward a more AI-driven, analytical, and scalable operating model. We are looking for interns to support day-to-day execution, improve data quality, and help build practical automation tools.
Interns will work with experienced team members on defined projects. These roles support delivery and experimentation, while full-time team members retain ownership of dataset strategy, quality-system design, model improvement decisions, and cross-functional alignment.

About the Role
Support the preparation, evaluation, and continuous improvement of datasets used for large language model (LLM) and AI agent development. This role combines structured data operations with hands-on exposure to model evaluation, error analysis, and prompt testing.

As a Project Intern, you will contribute to impactful short-term projects and gain hands-on experience in a fast-paced, professional environment. This internship offers the opportunity to develop practical skills, apply your knowledge to real-world challenges, and explore your career interests.
Applications are reviewed on a rolling basis, so we encourage you to apply early.

Responsibilities
- Prepare, clean, sample, and organize training and evaluation data according to defined requirements.
- Coordinate annotation tasks, track progress, and consolidate questions or disagreements for team review.
- Validate labels, metadata, and dataset completeness, and maintain version records and change logs.
- Organize model errors and representative cases into review-ready evidence packs to support root cause analysis (RCA).
- Run evaluation and prompt-testing workflows under guidance, record results, and compare outputs across iterations.
- Maintain operational trackers and recurring reports covering dataset readiness, annotation progress, and quality issues.
- Identify repetitive steps that could be simplified through templates, scripts, or AI-assisted workflows.

Qualifications

Minimum Qualification(s)
- Currently pursuing an Undergraduate in data science, computer science, statistics, information systems, social science, economics, or a related discipline.
- Strong attention to detail and the ability to follow structured guidelines consistently.
- Comfortable working with spreadsheets and organizing datasets and documentation.
- Basic understanding of LLMs and an interest in model evaluation and data quality.
- Clear written communication in English and the ability to manage tasks and escalate issues promptly.

Preferred Qualification(s)
- Experience with Python, SQL, data annotation, or dataset quality checks.
- Exposure to prompt engineering, model evaluation, or AI-assisted research projects.
- Interest in content governance or trust and safety.

Responsibilities

Qualifications

Qualifications

Minimum Qualification(s)
- Currently pursuing an Undergraduate in data science, computer science, statistics, information systems, social science, economics, or a related discipline.
- Strong attention to detail and the ability to follow structured guidelines consistently.
- Comfortable working with spreadsheets and organizing datasets and documentation.
- Basic understanding of LLMs and an interest in model evaluation and data quality.
- Clear written communication in English and the ability to manage tasks and escalate issues promptly.

Preferred Qualification(s)
- Experience with Python, SQL, data annotation, or dataset quality checks.
- Exposure to prompt engineering, model evaluation, or AI-assisted research projects.
- Interest in content governance or trust and safety.

About TikTok

TikTok is the leading destination for short-form mobile video and our mission is to inspire creativity and bring joy.

Size: 10,000+ employees
Industry: Technology: Consumer Internet

View company profile