The research process is the most basic element of scientific inquiry: Identify a puzzle in the existing literature, generate a hypothesis, create experiments to test it, analyze data, and draw conclusions that, with any luck, will later get published. But most undergraduates have never gone through that, at least not from start to finish.
That’s what William (Zev) Berger, a fellow in the University of Pennsylvania’s Philosophy, Politics, and Economics (PPE) program, aimed to provide through a two-semester research practicum. Fourteen juniors and seniors have now taken the course, which ran for the third time this past year. The most recent cohort studied the effect of “radical candor”—empathically delivered critical feedback—on mentorship outcomes. They plan to continue the work this summer.
“A course like this allows undergraduates looking for unstructured engagement to be able to choose their own adventure and create a research program for themselves,” says Berger. “To me, that’s how the best learning happens.”
Origins of the practicum
The class arose out of a requirement for Berger to add a second course to his teaching roster. He had broad subject-matter flexibility for the new offering, so he opted to build a hands-on experience centered around independent research.
“It’s one thing to learn from academic articles,” he says. “It’s another to try to replicate or implement what those articles did.”
He decided that keeping the group small—no more than five students—could facilitate the best outcomes, and he asked them to apply the spring before they would begin the class. They had to provide a resume and demonstrate their passion for studying social and political inequality. The aim, Berger says, was to ensure that students were both a good fit for the course’s rigor and internally driven to do research, as well as to find a diverse and passionate group of students who complemented one another.
During the first research practicum, which ran during the 2018-19 academic year, students looked at the relationship between social mobility and economic inequality. The second iteration focused on how inequality and risk tolerance affect people’s willingness to solve collective problems in the context of climate change. Most recently, the research project centered around the concept of radical candor mentorship.
The fall semester
Before delving into details of what the latest cohort discovered, it’s important to understand that for much of the first semester, each student did her own thing—intentionally.
“The fall was us trying to decide what we were going to ultimately do, learning about the research process and how to understand data, getting more experience with academic journals,” says Jeanica Geneus, a rising senior from Brooklyn who took the course during her junior year. “Between the five of us, we each had our own ideas, our own issues of inequality that we wanted to tackle.”
Berger encourages that kind of topic diversity at the beginning. It allows students to learn how to shape their own ideas, presenting an important pedagogical opportunity, where each student considers what manipulation—a factor that changes in one context but not another—might be used to assess a particular research question.
“Students come to the table having in mind a lot of correlational studies, where there’s a strong statistical association between two variables,” Berger says. “They tend to weave a causal account around that. But we need to be able to manipulate something in particular through a natural experiment or a lab study, and that’s the kind of causal framework the students have to wrap their minds around.”
In the fall, Berger also instructs on how to do a literature review, identify a puzzle or problem in the literature, and test a hypothesis. By the end of the first semester, each student has presented a project idea, and the group has voted on which it wants to pursue. As a team, the students write a research proposal on the agreed upon topic, and when they return in the spring they work on experimentation.
The spring semester
Starting a research program from scratch isn’t easy. That’s why Berger provides students two semesters to fit it all in.
In the spring, students “develop consent forms, pre-register the experiment, gather a sample,” Berger says. “They ultimately implement the experiment and write up the results, and their final paper serves as the nucleus of a research paper.” This year students also secured funding, receiving a $2,000 grant from the Andrea Mitchell Center and $1,500 from Campaign for Community through the Office of the Provost, on top of a $3,000 allocation from the PPE program.
Geneus’ cohort, which also included May graduates Joyee Au Yeung of Hong Kong, Elizabeth Benham of San Francisco, Lucia Vita of Oxshott, England, and Sarah Kim of Greensboro, North Carolina, focused on a question about mentorship and educational inequality. Specifically, the students wanted to understand whether radical candor might help students perform better.
Vita had heard about radical candor in another course she was taking, one with Penn professor Angela Duckworth. In that instance, Duckworth had talked about it in the context of management; Vita thought it might apply to education, too, so she decided to build her individual project around it. The students ultimately chose to move hers forward collectively.
“Then we started thinking about what the experimental procedure looked like,” Geneus says. They settled on using questions from the GMAT, the exam for graduate-level business and management programs.
Participants—all college-age students—initially completed 10 GMAT questions, then watched a short video of tips on how to answer them. Control group members then had a friendly chat with a mentor, and treatment group members received critical feedback on their performance. As a last step, everyone answered 10 more GMAT questions.
“The radical candor manipulation turned out to be consistently significant and important,” Berger says. “But surprisingly the effect was negative across all our models. It lowered the score of participants in the treatment group by roughly 1 point out of 10.”
It was the opposite of what the students had hypothesized, Geneus says. “Initially, we were a little stunned. We expected that radical candor mentorship would have a positive effect, leading to an increase in people’s performance,” she says. Eventually, “we realized that you can’t take shortcuts in trying to build genuine mentor relationships with people and in giving out radical candor feedback.”
Despite the unexpected results, the process did precisely what it had intended to do: It taught the five Penn students how to ask and answer a research question, even when that answer doesn’t turn out to be the hoped-for conclusion.
“This kind of process is incredibly messy. It’s not linear. It doesn’t unfold according to a predicable path,” Berger says. “But the upshot is that these students have now seen the difficulties. They’re better able to understand what’s going on behind the scenes. And they’ve learned for themselves how this is all done. Regardless of whether they go on to careers in research or industry, they’ll have a sophistication that sets them apart.”
William (Zev) Berger is a fellow with the Philosophy, Politics, and Economics program in the School of Arts & Sciences at the University of Pennsylvania. In July he will begin a position as an assistant professor at the University of Arizona. He is not related to the author of this article.
Jeanica Geneus is a rising senior from Brooklyn majoring in Philosophy, Politics, and Economics in the School of Arts & Sciences.
