Poster Session 3, 1:15 PM - 2:00 PM: Room 163 [C13]

Measuring and Evaluating AI-Generated Content on TikTok

Presenter: Jennifer Ye

Group Members: Manya Mehta, Tanish Gupta, Arshiya Sharma

Faculty Sponsor: Ethan Zuckerman

School: UMass Amherst

Research Area: Computer Science

ABSTRACT

As generative video technologies become increasingly accessible, AI-generated media is rapidly proliferating on social platforms such as TikTok, raising questions about platform disclosure policies  and the reliability of their automated labeling systems. This study examines the prevalence of AI-generated videos on TikTok and evaluates the performance of the platform’s AI-labeling mechanisms. Using a random sample of TikTok videos collected before and after the emergence of AI-generated video, we developed a technical pipeline that extracts video metadata, creator annotations, and engagement metrics from hundreds of videos. Rather than treating any single indicator as ground truth, we compare multiple signals of AI generation including creator-provided tags, platform-applied AI labels, and classifications produced by a high-accuracy independent visual detection model. By examining the agreement and divergence among these signals, we assess how reliably AI-generated content is identified on the platform. Using the resulting dataset, we analyze engagement outcomes, including views, likes, shares, and comments, to explore whether AI-related signals are associated with different patterns of visibility and interaction. The study provides an empirical evaluation of TikTok’s AI disclosure ecosystem and contributes evidence to broader discussions of accountability, trust, and governance in rapidly evolving social media systems.