Crime and the scientific method

From the procedurals and documentaries that populate streaming services to the oppositional catch phrases that dominate political debates, crime is a constant theme in our national discourse. But beyond entertainment and politics, there is the reality of crime, and understanding this reality is where the Department of Criminology in Penn’s School of Arts & Sciences comes in.

“Penn, going back a hundred years, has been in the mode of good data collection and scientific method in studying criminology,” says Greg Ridgeway, professor and department chair, who also holds an appointment in the Department of Statistics and Data Science at the Wharton School. “The key components of doing a science experiment are developing good controls, taking measurements with the least amount of error and the most amount of precision, and producing research that’s reproducible that actually gets at a causal estimate rather than just an observable association. As a department we’re really passionate about this.”

Better policing through social science

This “scientification” of criminology, as Ridgeway puts it, carries across a range of projects undertaken by the interdisciplinary faculty of the department, from analyzing the propensity of police officers to fire their guns to examining street lighting’s effect on crime. 

“Science would say that if you are a chemist, biologist, or physicist, you would conduct a controlled experiment,” says Ridgeway, who worked at Microsoft Research in the late ’90s, where he helped to pioneer algorithms that would recommend content to users—work reflected in seven patents that he holds. “For example, we can’t necessarily randomize where to locate a homeless shelter, but in the city of Vancouver, each winter, they relocate their emergency winter shelters, so we can take that information and look at what happened before, during, and after the shelter was present at any given location, and then use that information to try to understand issues of public safety around those places.” This can inform policy and planning.

Large datasets—prosecutorial records, police misconduct histories, gun violence rates, and more—are an invaluable resource for modern criminologists, and Ridgeway’s own research explores the use of big data to improve policing. In one such project, Ridgeway worked with the New York Police Department to analyze police shooting incidents. “What I wanted to know is, you’ve got two officers on a scene and one of them shoots and one doesn’t—what’s different about them?” says Ridgeway. “They’re both looking at the same offender, the lighting’s the same, they’re in the same environment, but for some reason, one decided ‘I got to shoot,’ and the other one didn’t.”

The first step, Ridgeway continues, was to find incidences of a pair of officers in this particular scenario, then pull all their information, including age, race, and sex. “But also, when did they join the department? When was the last time they went to the range to shoot? What was their performance review like? Have they been injured on the job? Had they crashed cars? Did they have lots of negative marks in their file? Had they received medals of valor? I kept pulling all these data points to see what was different between these officers, and because I was able to do that, I uncovered that the officers who accumulate lots of negative marks in their file are three times more likely to be shooters.”

Ridgeway is conducting a similar study in another major police department where he aims to enumerate individual officer’s personal risk of escalating force. “I’m able to pull a lot of information because police departments are collecting a lot of information and putting it in formats where you know what happens when and who was involved,” he says. “Lots of information is being collected and somewhere in there is a piece of information that’s finally being collected in a way that allows you to answer an important question.”

Aaron Chalfin, an assistant professor in the department, also works to advance research methods—in his case, focusing on measurement problems. He then applies these methods to issues surrounding policing. For instance, Chalfin has analyzed police misconduct data to examine the “bad apple” theory. “Some popular media accounts report that 3% of the officers commit 50% of the misconduct, which suggests, ‘If we could only just get rid of that 3% percent, those few bad apples, we could get rid of a huge share of the problem.’ It sounds like a panacea solution, which really doesn’t exist in public policy.”

Chalfin says that if you look back at an officer’s career, risk is much easier to track. But trying to weed out bad officers during their probationary period—before it becomes much more difficult to fire them—is another story. “We think that if you got rid of the top 10% of high-risk officers at the end of their probationary periods it might eliminate 5 to 8% of the misconduct—not 50%,” says Chalfin. “Plus, you need to allow for the replacement of the potential bad apples with other officers, who themselves will commit some misconduct on average.”

Unequal impacts

Another important aspect of the department’s research is to understand the impact the criminal justice system has on the lives of people involved in it. Charles Loeffler, an associate professor in the department, researches the effect incarceration has on the likelihood that people will have continued justice system involvement. “I spend a lot of time trying to understand better the impact of policy changes that have shifted the boundary between the juvenile and adult justice systems,” says Loeffler.

Loeffler also collaborated with Ridgeway on a project examining wrongful convictions. The predictable question they faced: How likely was it that inmates who claimed they were wrongfully convicted were telling the truth?

“We found many individuals who had been convicted, say, five times, and admitted to being guilty in four of the five of them. This indicates that they are perfectly willing to admit guilt in some cases,” says Ridgeway. “Charles’ insight was to try to separate the ‘always claim innocence’ individuals from the ‘willing to admit guilt individuals.’ I then developed a statistical model to actually do the calculations. In the end, 8% of survey respondents said that they were innocent, but with Charles’ insight and my model, we think that about 2% of them would claim total innocence no matter what. This means the ‘true innocence rate’ is likely around 6%—still quite different from zero.”

In another example of analyzing unequal impacts, Aurélie Ouss, the Janice and Julian Bers Assistant Professor in the Social Sciences, focuses on how criminal justice institutions and policies can make law enforcement fairer and more efficient. Many times, this translates into the examination of oft-debated practices, like progressive prosecution, which advocates for fewer prosecutions of low-level offenses and increased diversion and treatment programs.

“Prosecutors’ decisionmaking has been described as the ‘black box’ of criminal justice,” says Ouss, whose Ph.D. is in economics. “Many hypothesize that they are very influential because of their discretionary power, and that they play an overlooked role in the growth of incarceration. But there is little data on what they do.”

Ouss, who along with Ridgeway has been collaborating with the Philadelphia District Attorney’s office to examine what kinds of decisions prosecutors make and how they affect the outcomes of cases, is researching cash bail policy to determine whether it does more harm than good. “A lot of the cash bail conversation has been focused on its impacts on pretrial detention,” Ouss says. “But many people are released pretrial with financial collateral, and there is little research on what impact monetary bail has for these people.”

So, why have cash bail in the first place? Ouss says it is centered around the idea that people will have monetary skin in the game and therefore be more likely to appear in court when a dollar amount is attached to their appearance. But what she found was that a decrease in cash bail had no effect on failures to appear, or on pretrial crime.

“If you work the cash bail problem backward, a negative effect of cash bail is that poor people are more likely to be incarcerated,” says Ouss. “However, cash bail does not fill its intended purpose of increasing court compliance—as opposed to text message reminders, which I’ve shown to be very effective. To design better criminal justice policies, we need to rethink why people don’t comply.”

Ouss is working on this question with a number of graduate students, including Viet Nguyen, a doctoral student in criminology, whose research interests include corrections policy and criminal justice reform.

“Criminal cases are complex, and prosecutors work under different constraints that are generally unobserved in the data,” says Nguyen. “Conducting court observations and having meetings where prosecutors walk me through their thought process has been immensely helpful. These prosecutors have helped me understand why a case may receive a certain disposition beyond traditional measures like the seriousness of the offense at hand.”

Violence patterns

One of the biggest concerns in the study of crime, and in public safety, is the rise in gun violence. Loeffler’s work examines this problem as well, and he discusses what he calls a competing hypothesis. “Some scholars say, ‘Look, gun violence is contagious, and that violence breeds more violence, so we need to be in the business of trying to interrupt those cycles of violence.’ But another explanation focuses much more on the role of random disputes that lead to violence.”

Loeffler, whose resume also includes the invention of a wearable Fitbit-like device that can detect whether the wearer has fired a gun, says there has been a tension among researchers in regard to which better explains the extent of gun violence. In response, he and colleagues statistically analyzed the relative contribution of these two problems. They concluded that both explanations account for a portion of the violence, but that the ‘contagion’ aspect represents a minority.

“It’s not a trivial amount by any manner, but this tendency to view these competing hypotheses as either-or misses the point that it’s more of an ‘and,’” says Loeffler. 

“A lot of communities experiencing substantial gun violence problems have been experiencing substantial gun violence problems for years,” he adds. “And yet we can see these fluctuations that are fed by more temporary surges in retaliatory violence. This creates a situation where we need to employ statistical models to allow us to better understand the patterns that we do see.”

The science of evidence

Assistant Professor Maria Cuellar also applies a statistician’s toolkit, by both investigating error rates in forensics and creating new methods of analysis. “Even today, but more commonly before the use of DNA, analysts claimed that by observing hair with a light microscope, they could tell the race of the person who left that hair,” says Cuellar, whose Ph.D. is in statistics and public policy. “But when you look at the data, it turns out that you cannot really distinguish race based on microscopic hair evidence. In my research, I point out problems in the way that arguments are made and show that these could be leading to errors in the criminal justice system, both in terms of wrongful convictions, but also in miscarriages of justice.”

Cuellar is developing new forensic analysis methods, starting with toolmark analysis. “Toolmarks are the marks left in a crime scene by a handheld tool on a surface,” says Cuellar, who also holds an appointment in the Department of Statistics and Data Science at the Wharton School. “For instance, if someone breaks into a house, there might be some marks left on a doorframe, and if an explosive device is detonated, there are marks left on the wires or the pieces of metal found in the debris. There might be a suspect who owns some tools, so the task of forensic toolmark analysts is to compare the marks left by the suspect’s tools versus the marks left at the crime scene.”

View large image

Analyzing toolmarks is similar to analyzing a bullet used in a shooting, but it’s even more challenging. “With a firearm, there’s the ammunition and there’s the trigger,” says Cuellar. “There is not much variation in how you shoot the firearm. But with tools, if you imagine using a knife to pry a door open, there are so many different ways in which the knife could be used, and these many different degrees of freedom make it a lot more difficult to study toolmarks.”

To accomplish this, Cuellar makes toolmarks using a mechanical rig to ensure controlled, consistent marks. The rig can adjust the force and angle so that she can study the variations. “Right now, I’m working with screwdrivers, which are one of the simplest tools you can have,” says Cuellar. “It just has two surfaces, and you can make striation marks that are just drag marks on metal. I then scan those with a 3D scanner and use that data for analysis. I’m interested in starting with the simplest tool and build up from there. The most important element is creating high-quality data from known sources, known as labels, that you can use to train the algorithm.”

Cuellar says that some are skeptical of new methods. “Many critics say, ‘Well, how can you study toolmarks? They are so complex that really it just should be done by a human. It’s more of an art than a science,’” says Cuellar. “And what I’m saying is that statistics is the perfect tool to study problems like these. They are our bread and butter. With careful modeling and machine learning, we can help improve the quality of forensic analyses and reduce errors in the criminal justice system.”

A policy focus 

One of the criminology department’s main priorities is research that leads to meaningful reform, whether it’s researching ways to make the justice system more equitable or developing strategies to strengthen communities. The Crime and Justice Policy Lab, led by Anthony Braga, the Jerry Lee Professor of Criminology, dedicates its resources to high-impact initiatives that make a difference. For instance, the lab recently completed a randomized experiment that showed police training in procedurally just encounters can reduce crime and improve police officers’ relationships with the communities they serve.

The lab partners with governments and nonprofits to find research-based solutions to preventing crime, improving the justice system, and other complex social problems, with a focus on the needs and priorities of the partners and communities that lab members work with. 

To this end, in an ongoing project in Baltimore, Braga and the lab’s executive director Ben Struhl have been working closely with the mayor on a comprehensive violence-prevention plan.

“A lot of efforts to address crime and violence are incredibly ineffective because there is a very democratic impulse to take resources and spread them as wide as possible,” says Struhl. “But if resources are spread as wide as possible, that means they’re also spread as thin as possible. By forming the city partnership with Baltimore, we were actually able to get them to focus in on their strategy, and make sure that what they were planning to do was based on scientific evidence.”

To create an effective plan of action, Braga says, you need to customize the strategy to local conditions and local capacities, which requires upfront research to describe the problem and logically link the intervention. To facilitate this process, Braga and Struhl have regular meetings with the mayor and bimonthly meetings with the deputy mayor.

“It takes time to line up the criminal justice agencies required to implement an evidence-based plan,” says Braga. “This process also includes social service agencies responsible for providing opportunities to those who want and need them in the offending population that’s at the heart of most violence within cities, as well as being able to mobilize the community effectively behind an anti-violence message in a coherent way.”

Another of the lab’s projects aided Boston’s leadership in formulating an effective response to the city’s opioid crisis. The lab spent two years doing extensive interviews with people with opioid use disorder (OUD), completing systematic observations of the sites where selling and buying activity was occurring, and mapping out and talking to all the relevant parties—law enforcement and homeless shelters, for instance—that were part of the dynamic.

“We found a real gap where people who are suffering from OUD really are repeatedly victimized and incredibly unsafe,” says Struhl. “They’re very fearful, but at the same time they don’t want to interact with law enforcement because they’re worried law enforcement will hassle them or take away their drugs, which they’re addicted to. But there’s actually a lot that we identified that could be done to prevent harm to this population.”

The lab used innovative social network analysis techniques to examine risks of overdose and found that one network represented only 1% of the city’s population but accounted for more than 20% of all overdoses in the city. “This led us to ask if it was somehow possible to make opioid use within that network less harmful, less risky, less dangerous, given the extensive concentration of overdoses within this very small group of connected people? And we developed a series of ideas and interventions,” says Braga.

Addressing the problem at its source

Being situated in Philadelphia provides faculty a unique opportunity to observe and engage in communities that experience high levels of crime. The city’s crime rate ranks above the national average in regard to violent offenses, and poverty rates are high. “There are lots of opportunities to collaborate with nonprofit criminal justice agencies and discuss issues with policymakers,” says John MacDonald, a professor in the department. “The work I’ve done on topics such as vacant housing remediation ends up being brought up a lot more around city council, whereas if I was a faculty member at a different university, it probably wouldn’t have the same weight.” 

MacDonald’s research asks, what if living conditions—variables like light, temperature, and tree cover—play more of a role in our safety than we ever imagined? MacDonald and colleagues recently completed a six-year research project that randomized abandoned houses on some blocks to receive remediation, including the graffiti cleaning, trash removal, and the installation of working doors and windows. They found significant effects across the board in reductions in aggravated assaults with firearms. “This suggests keeping residences up to city code by putting in working windows and doors seems to at least bring down some of the violence,” says MacDonald. “People are living in places that are falling apart and we need major investments. There’s now increasing science to show that even if you don’t necessarily provide direct services to people, just make the conditions where they’re living better.”

View large image

MacDonald has also investigated temperature’s effect on violence, specifically how summer heat can lead to high-stress situations, and in turn, violence. “There’s general consensus from a number of studies that finds when you have more people out and about, there is less street crime,” he says. “There are more stores open, more eyes on the street. During a heat wave there are fewer people out policing each other. There also tends to be increased alcohol consumption, which leads to a perfect storm of bad judgment. And communities that are resource-deprived and don’t have access to air conditioning, or a shopping mall to go to in order to avoid the heat, are under more intense community pressure, and crime is a natural consequence.”

MacDonald has also researched how tree death and crime correlate, using coordinates provided by the Forestry Service and comparing them with crime reports. He found that, across the board, most major categories of crime went up after trees died as a result of phenomena like seasonal foliage-killing beetles. “There are many hypotheses. For instance, if you have trees with canopy on them, they may block visibility into homes, which is a natural deterrent to possible intruders,” says MacDonald. “This theory is given credence by the fact that property crime turned out to have the highest increase when trees were absent.”

Like MacDonald, Chalfin has also studied the effects of physical environment on crime; he worked with the City of New York to chart the effect of adding light towers to a public housing community. “We found that over the initial six months that the lights were there, outdoor nighttime crime was down by about 36%,” says Chalfin. “They were supposed to be taken away after six months, but it turns out they stayed because people in the neighborhood didn’t want them to go away.” A three-year follow-up revealed that crime reductions remained. 

Adrian Raine, the Penn Integrates Knowledge Richard Perry University Professor with appointments in Penn Arts & Sciences and the Perelman School of Medicine, also looks at the complex dynamics between environment and crime, but as an expert in neurocriminology, he focuses on what contributes to violent behavior on an individual level.

Raine has been conducting research in Africa since 1980, where he studies child health and development, trying to understand how factors in early childhood might encourage criminal behavior later in life. In an early-intervention study in Mauritius, an Indian Ocean island nation, Raine and his team enriched the environment of three-year-old children for two years, providing them better nutrition, more physical exercise, cognitive stimulation, and regular naps. These children were then matched with others from Mauritius who did not receive any intervention.

“I followed up for 20 years to find out which ones became criminal offenders, and I found that the enrichment early on in life reduced criminal offending 20 years later by 34%,” says Raine. “Now, it’s not a magic bullet—it’s not wiping away crime. But nevertheless, it is making a dent. So, for me, I think one of the best investments society can ever make to stop crime is investing in the early years of the child.”

Raine’s team measured brain function in the same children using electroencephalography and found that the kids who had gone into the enrichment program had more alert and aroused brains. “Their brain had matured 1.1 years more than the kids who never had the enrichment,” says Raine. “So, the enrichment is enhancing the brain, and if you enhance the brain, you enhance behavior.”

Raine has also done extensive research on neurological health and how it informs behavior. He recently oversaw a study by then-graduate student Olivia Choy, now an assistant professor at Nanyang Technological University in Singapore, in which she examined whether an impairment in the part of the brain responsible for aggressive behavior, impulse control, and emotion regulation led to aberrant behavior. 

“Our thinking was, if we can enhance that part of the brain, maybe that will reduce aggressive behavior,” says Raine. In collaboration with Penn’s Neurology Department, they use electrodes to stimulate and “upregulate” the brain’s prefrontal cortex. “In a randomized control trial, we found that this type of stimulation reduced the intention to commit an aggressive act and enhanced a person’s moral sense. As I see it, the future of neurocriminology lies in better understanding how the social environment gives rise to biological risk factors for crime.”

Whether by examining data and statistics, working with government officials and police to forge strong bonds within communities, spearheading policy reform, or helping the next generation achieve positive outcomes, Ridgeway says pulling together all these diverse scientific disciplines will help answer the tough questions—and ultimately make safe communities. “It takes a special group of students, staff, faculty, and partners in communities to make this happen.”