The Programme for International Student Assessment (PISA) is issued every three years in more than 80 countries, examining how 15-year-old students perform in reading, mathematics, and science. Developed by the Organisation for Economic Co-operation and Development, or OECD, it launched in 2000 and offers comparative data that countries can use to influence education policy.
Jonathan Supovitz, a professor of policy, organizations, leadership, and systems division in the Graduate School of Education, teaches a course on assessment and is the director of the Consortium for Policy Research in Education. Supovitz says the overall value of PISA and other international exams, such as the Trends in International Mathematics and Science Study, is that they serve as catalysts for change—though they do come with limitations.
“The upside is you get a sense of how your system is doing relative to other systems, and the downside is that it’s so detached from the actual things that schools and districts do, particularly in a system as decentralized as the U.S., that it makes it hard to make general statements about the patchwork of education that makes up the U.S. education system,” says Supovitz.
Participating countries range from Japan to Switzerland, the United States to portions of China that include Macao and Hong Kong. Students take a two-hour exam and then respond to a 30-minute survey. Approximately 690,000 students globally completed the exam in 2022; countries typically assess between 4,000 and 8,000 students. Fifteen-year-olds are selected in particular, because it is the last consistent age at which students are internationally enrolled in formal education.
Below, Supovitz offers five takeaways from this year’s PISA results, released earlier this month.
Quality of time spent learning math matters
“Somewhat surprising about math is that it’s not the amount of time we spend on learning, because they also measure the amount of time kids were in classes in math and other subjects, but the amount of time isn’t related to achievement,” Supovitz observes. “It’s the rigor and ambition of how we do spend that time, and the structure of the curriculum.”
In this sense, results can vary by school—and even within schools, depending on their structures and policy.
“I think this has to do with how we are giving differential attention and how we are allocating teachers and resources within districts and within schools,” says Supovitz. “In research, we talk a lot about the distinction between the variation that’s within schools and across schools, and in the PISA performance, the variation within countries was much greater than the variation across countries, and this is also true within schools.”
Broadly, Singapore scored the highest in math—by a significant margin—and was followed by Hong Kong, Japan, Korea, and Macao. (Singapore also scored the highest in reading.)
Countries that opened schools sooner during the pandemic performed better
The U.S. appears to be keeping pace with reading and science but is lagging in math. The PISA results demonstrate that the U.S. had lower-than-average math performance compared to countries that closed schools for a shorter duration during the pandemic. This is a trend among all countries who made this policy decision.
Supovitz says it may also reflect how students learn at home.
“When you think about parents’ ability to support subjects, in general people feel less capable of supporting math outside of school than they do reading,” he says. “We can all have our kids read or read to them, but it takes more knowledge and skill of teachers to bring kids through the math trajectories of learning, and so I think this really does say something about the value the education system has been adding for kids that has been unacknowledged for a long time.”
Technology is a distraction
“A new and emerging finding coming out now is that technology use out of school is potentially a major distraction and clearly linked to lower outcomes,” says Supovitz.
This was true not just of time spent learning during the pandemic, but also in the classroom. More and more schools in the U.S., Supovitz says, are contemplating outright bans of phones in the classroom.
“Kids today are so attached to their phones, and there are some interesting findings in PISA that kids feel nervous or anxious when their digital devices are not near them,” says Supovitz. “And higher levels of outside of school technology use by kids were related to lower scores on PISA,” he adds.
This is true across all countries who have a high level of access to digital devices for leisure time, not just the U.S.
Equity didn’t worsen; affluence doesn’t guarantee strong performance
“Another piece of good news is that we might have thought the pandemic would have exacerbated inequities, but because of this longitudinal perspective you can get from PISA, while we didn’t reduce inequities, we didn’t expand them either,” Supovitz says. “They were fairly stable from 2018, before the pandemic, to 2022.”
An alarmingly few disadvantaged American students—about 11 percent—land in the top quartile of math scores, for example, but even affluent U.S. students did not match average-performing students in Japan, South Korea, and Hong Kong, and it’s notable that Americans in general were among the most advantaged students taking the 2022 PISA test.
The socio-economic gap in math did not change in 51 out of 68 countries.
Results only matter to the extent we can talk about them
Supovitz recalls that past results have led to discussions about, for example, setting national Common Core standards. And while the debate is worthwhile, it’s often politically volatile and a reminder that every education system exists in context.
“I think that [PISA] sparks really good conversations and really calls attention to some of the choices that we’ve made—even though we might not think of them [as choices] because they’re so embedded in the way we do things,” says Supovitz. “So, this does offer a refreshing opportunity to examine some of the assumptions of our system and say, ‘This is, for better and worse, what we’re producing,’ and if we truly do want to improve our system then these are some contrasts from the way others do things.”
He adds that the results are most useful if there’s a will to make a change.
“In some ways, this is a reflection of our education system, but in other ways, it’s really a reflection of our society,” he says. “I don’t think you can separate our education system from the way we’ve constructed our social system and culture.”
