A new study from Annenberg School for Communication’s Computational Social Science Lab finds that the YouTube recommendation system is less influential on users’ political views than is commonly believed.
About a quarter of Americans get their news on YouTube. With its billions of users and hours upon hours of content, YouTube is one the largest online media platforms in the world.
In recent years, there has been a popular narrative in the media that videos from highly partisan, conspiracy theory-driven YouTube channels radicalize young Americans and that YouTube’s recommendation algorithm leads users down a path of increasingly radical content.
However, a new study from the Annenberg School for Communication’sComputational Social Science Lab (CSSLab) finds that users’ own political interests and preferences play the primary role in what they choose to watch. In fact, if the recommendation features have any impact on users’ media diets, it is a moderating one.
“On average, relying exclusively on the recommender results in less partisan consumption,” says lead author Homa Hosseinmardi, associate research scientist at the CSSLab.
YouTube bots
To determine the true effect of YouTube’s recommendation algorithm on what users watch, the researchers created bots that either followed the recommendation engines or completely ignored them. To do this, the researchers created bots trained on the YouTube watch history from a set of 87,988 real life users collected from October 2021 to December 2022.
Hosseinmardi and co-authors Amir Ghasemian, Miguel Rivera-Lanas, Manoel Horta Ribeiro, Robert West, and Penn Integrates Knowledge Professor with appointments in the School of Engineering and Applied Science, the Annenberg School for Communication, and the Wharton SchoolDuncan J. Watts aimed to untangle the complex relationship between user preferences and the recommendation algorithm, a relationship that evolves with each video watched.
These bots were assigned individualized YouTube accounts so that their viewing history could be tracked, and the partisanship of what they watched was estimated using the metadata associated with each video.
User behavior versus algorithm influence
During two experiments, the bots, each with its own YouTube account, went through a “learning phase”—they watched the same sequence of videos to ensure that they all presented the same preferences to YouTube’s algorithm.
Homa Hosseinmardi, associate research scientist at the Computational Social Science Lab, and Duncan Watts, Stevens University Professor and director of the Computational Social Science Lab.
(Image: Courtesy of Annenberg School for Communication)
Next, bots were placed into groups. Some bots continued to follow the watching history of the real life user it was trained on; others were assigned to be experimental “counterfactual bots”—bots following specific rules designed to separate user behavior from algorithmic influence.
In experiment one, after the learning phases, the control bot continued to watch videos from user’s history while counterfactual bots deviated from users’ real-life behavior and only selected videos from the list of recommended videos, without taking the user preferences into account.
Some counterfactual bots always selected the first (“up next”) video from the sidebar recommendations; others randomly selected one of the top 30 videos listed in the sidebar recommendations; and others randomly selected a video from the top 15 videos in the homepage recommendations.
The researchers found that the counterfactual bots, on average, consumed less partisan content than the corresponding real user—a result that is stronger for heavier consumers of partisan content.
“This gap corresponds to an intrinsic preference of users for such content relative to what the algorithm recommends,” Hosseinmardi says. “The study exhibits similar moderating effects on bots consuming far-left content, or when bots are subscribed to channels on the extreme side of the political partisan spectrum.”
‘Forgetting time’ of recommendation algorithms
In experiment two, researchers aimed to estimate the “forgetting time” of the YouTube recommender.
“Recommendation algorithms have been criticized for continuing to recommend problematic content to previously interested users long after they have lost interest in it themselves,” Hosseinmardi says.
During this experiment, researchers calculated the recommender’s forgetting time for a user with a long (120 video) history of far-right video consumption who changes their diet to moderate news for the next 60 videos.
While the control bots continued watching a far-right diet for the whole experiment, counterfactual bots simulated a user “switching” from one set of preferences (watching far-right videos) to another (watching moderate videos). As the counterfactual bots changed their media preferences, the researchers tracked the average partisanship of recommended videos in the sidebar and homepage.
“On average, the recommended videos on the sidebar shifted toward moderate content after about 30 videos,” Hosseinmardi says, “while homepage recommendations tended to adjust less rapidly, showing homepage recommendations cater more to one’s preferences and sidebar recommendations are more related to the nature of the video currently being watched.”
What’s next
“The YouTube recommendation algorithm has been accused of leading its users toward conspiratorial beliefs. While these accusations hold some merit, we must not overlook that users have a significant agency over their actions and may have viewed the same content, or worse, even without any recommendations,” Hosseinmardi says.
Moving forward, the researchers hope that others can adopt their method for studying AI-mediated platforms where user preferences and algorithms interact, in order to better understand the role that algorithmic content recommendation engines play in our daily lives.
In Senegal, the ambitious Dakar Greenbelt project seeks to create an extensive network of ecological infrastructure in and around the city to sustainably address environmental concerns and enhance urban life. With support from David Gouverneur and Ellen Neises, Ph.D. candidate Rob Levinthal in the Weitzman School of Design led two courses that included a field trip to Dakar, that culminated in students presenting their visions for parts of the Greenbelt.
From a desert to an oasis: Penn engages in ambitious greening effort in the Sahel
Students from the Weitzman School of Design journeyed to Senegal to help with a massive ecological and infrastructural greening effort as part of their coursework. The Dakar Greenbelt aims to combat desertification and promote sustainable urban growth.
As part of an undergraduate course, Penn faculty and students curated an Arthur Ross Gallery exhibition of works from the Neumann family’s extensive collection of modern and contemporary art.
The University’s nexus for technology transfer supports researchers in their innovative efforts, from CAR T to mRNA advancements that have dramatically reshaped the world.
