When it comes to assessing how international organizations operate, the standard of measurement for them is remarkably less complicated than the organizations being measured.
If they’re being assessed at all.
International organizations are loosely defined as multilateral treaties between a group of nation-states that can either be global, like the World Health Organization, or more regional, like the Arab League. These organizations, says Julia Gray, an associate professor of political science in the School of Arts & Sciences, have not in the past been subject to much analytical research. The most broadly cited research, she says, is a dataset collecting information from 1816 to 1965 and was originally collected by political scientists Michael Wallace and J. David Singer in 1970. That dataset, which has since been added to, tracks the membership in, and existence of, these organizations from 1815 to 2014, but does not offer a detailed view of how active or productive they are—only that they exist or are active at all.
“I thought that wasn’t very useful, because we know international organizations change and evolve over time. Sometimes they go forward, sometimes even backward in dealing with the challenges they face,” Gray says. “There weren’t very good metrics out there that compare how organizations work on a dynamic basis, and so that was the original idea of this project.”
Gray’s research began in earnest in 2016, examining what international organizations do over their lifespan and finding ways to code them. Of the approximately 400 international organizations in the world, her research homed in on 75 that were focused on economic cooperation, her main area of interest. But as she realized many of these organizations had preceding or spin-off organizations, she expanded the scope of the research. She also wanted to further understand how international interactions can stem from regional interactions, which would emphasize the prominence of regional international organizations in different parts of the world.
To help with the project, she first enlisted Penn Undergraduate Research Mentoring Program (PURM) students in 2017. The program, which is part of the Center for Undergraduate Research & Fellowships, awards first- and second-year undergraduate students with funding to participate in faculty-led research. This summer, four students have assisted Gray with research, using Nexus Uni to comb through articles, meeting notes, press releases, and more to create a quantitative means to assess the activity of international organizations.
Rising sophomore Naveen Farook, an international relations major from Florida, was assigned a collection of international organizations ranging from 2014 to 2020. She says she scours them for accomplishments like new agreements signed, workshops, and other activities that are then placed on a timeline and plugged into a spreadsheet. That qualitative information is then turned into quantitative data to be used to assess the performance of organizations.
“For me, what I feel I’m getting out of the experience, is seeing what effective methods there are for these organizations in tackling certain issues or agendas,” Farook says. “A lot of organizations put out a lot of goals they want to achieve, and oftentimes it takes them several years to make a dent into where they want to be. So, I think that, with this project, I’m really getting the closer look at what it could take in order to achieve these things and how important it actually is to meet frequently.”
One organization she tracked was Mercosur, a South American trade bloc that primarily includes Argentina, Brazil, Uruguay, and Paraguay. In that case, she says, hundreds of articles up until 2019 cited the organization’s preliminary success in establishing a free trade agreement with the European Union, only for the deal to be put on pause over environmental concerns related to deforestation and fires in the Amazon. Internal strife within Mercosur has since shifted perception of what was previously considered a successful organization of South American powers.
“It says a lot about how much goes into creating alliances,” Farook says. “Whether it’s a trade agreement or some sort of political relationship.”
Farook says this is her first experience with research and her work has given her a clearer understanding of what research looks like in the social sciences, observing that people often associate research with more traditional science fields. She says she’s also enjoyed working with a diverse group of students from around the world and being given flexibility by Gray to pursue her interests.
Hannah Norman, meanwhile, is a rising sophomore from Virginia who was inspired to participate in the internship because of her experiences in Cambodia, where she lived for eight years. There, as a teenager, she frequently heard about international organization operations and was interested in comparing their effectiveness.
“In the States, I think it’s a common perception that international organizations are fairly ineffective, which is a case-by-case basis,” Norman says. “Regional ones make fairly good progress. So, I was interested in how that perception came about and how accurate it is.”
Her research involved examining a variety of organizations, much like Farook, as well as looking at how legal immunities given to various international organizations impact their effectiveness.
Norman says she saw the internship as an opportunity to experience what research looks like in the social sciences, and specifically understanding how important quantitative skills can be to conducting such research. She adds that the internship moved her to consider declaring a major in international relations.
The larger project, Gray says, will culminate in a series of publications that she hopes will be useful for scholars of international organizations. She also hopes it will serve as a public good, to better know what problems international organizations are good at solving and which they are not.
The PURM experience, she notes, has consistently been a good one.
“I love the PURM program so much,” she says outright. “I appreciate the opportunity to work with students I wouldn’t have otherwise connected with.”
She adds: “I just hope that the students get contacts and confidence in terms of not just how social scientific research is conducted, but more generally, thinking about their career trajectories and seeing how this works. I hope it will help them not just with their studies at Penn, but beyond.”
