Search Operation Quality Assurance for AI Model Evaluation

us.

About Trust & Safety
Building a world where people can safely discover, create and connect. The Trust & Safety (T&S) team at TikTok helps ensure that our global online community is safe and empowered to create and enjoy content across all of our applications. We have invested heavily in human and machine-based moderation to remove harmful content quickly and often before it reaches our general community.

About the Team
The Search Operations team aims to improve search user experience, contribute significant DAU impact to the products and drive for increase in traffic and GMV.

The T&S evaluation and investigation team, working closely with the search team, taking the evaluation standards as the baseline and principles, and manually evaluating the search query and the corresponding search results page to reflect the real user experience of the search function.

The team is committed to providing search and evaluation services for international products, with resource support provided by different countries around the world, and global members conduct evaluations according to unified guidelines. The evaluation scope covers multiple search sections, such as basic search experience, recommended search, search advertising, and search security. As a bridge for business communication domestically and abroad, the team is responsible for key nodes such as standard writing, training Q&A, report output, and human efficiency management to effectively promote the execution and implementation of projects.

It is possible that this role will be exposed to harmful content as part of the core role/as part of project/ in response to escalation requests/by chance.

This may occur in the form of images, video, and text related to every-day life, but it can also include (but is not limited to) bullying; hate speech; child safety; depictions of harm to self and others, and harm to animals.

What will I be doing?

• Complete regular assessments of AI model's quality in accordance with evaluation standard documents, based on business requirements
• Plan, conduct and support policy training and go-live processes for in-house specialists. Coordinate and support in-house specialists' tasks and data delivery
• Ensure overall data delivery quality and improve project performance to achieve established targets:
• Conduct quality inspection of evaluated data and specialists' daily tasks, comb the types of AI model bad cases
• Be responsible for standard document clarifications to ensure better understanding for in-house evaluation specialists
• Conduct calibration meetings with the team to analyze and align on different cases, evaluate performance, offer solutions and suggestions to improve accuracy rates
• Work and interact closely with POC and other stakeholders to provide timely and quality feedback on SOP/guidelines/data quality to optimize existing processes and to ensure overall project success
• Work on various evaluation-related projects outside core responsibilities based on business needs
• Perform other ad-hoc duties on relevant projects as assigned

Qualifications

What should I bring with me?

Minimum Qualifications

• Bachelor's Degree or above.
• About 2 years work experience in QA/data analysis/project management
• Excellent language and communication skills: proficient Korean and English at all levels (reading, writing, listening, speaking). Strong interpersonal skills, excellent attention to detail, precision and accuracy
• QA and quality improvement experience, familiarity with RCA and data analysis
• An in-depth understanding of local culture and familiarity with the Internet usage habits of local users
• Your ability to work in a high tempo environment, adapt, respond to day-to-day challenges of the role.
• Your resilience and commitment to self-care to manage the emotional demands of the role.

Preferred Qualifications

• Major in journalism, literature, mathematics, computer science, medicine, law, economics are preferred
• Overseas work/training experience or in a multi-cultural/multilingual environment is preferred
• Knowledge about AI or related models and search engines, familiarity with social media platforms

Trust & Safety recognises that keeping our platform safe for TikTok communities is no ordinary job which can be both rewarding and psychologically demanding and emotionally taxing for some. This is why we are sharing the potential hazards, risks and implications in this unique line of work from the start, so our candidates are well informed before joining.
We are committed to the wellbeing of all our employees and promise to provide comprehensive and evidence-based programs, to promote and support physical and mental wellbeing throughout each employee's journey with us. We believe that wellbeing is a relationship and that everyone has a part to play, so we work in collaboration and consultation with our employees and across our functions in order to ensure a truly person-centred, innovative and integrated approach.

TikTok is dedicated to fostering an inclusive and diverse workplace that values and respects individuals with disabilities. Our commitment is to create equal opportunities and provide a supportive environment for all employees.
TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.