An Inca ceremonial center, recreated in a digital landscape

From TV shows and movies with massive CGI budgets to state-of-the-art video games that take players into different worlds or time periods, many of today’s stories are told in a digital landscape. But computer graphics alone cannot tell a great story. A single uncanny face in bad CGI can take a person out of a scene, while a film that uses types of objects from the wrong time period can focus the audience’s mind on the mistake instead of the story.

As one example of efforts to unite digital technologies and the humanities, computer scientist Norman Badler and archaeologist Clark Erickson collaborate on summer research projects and teach a cross-listed course called Visualizing the Past/Peopling the Past. By working across these two fields, students can see firsthand how technology can help bring historical tales to life while also understanding the importance of anthropological context in effective and accurate storytelling.

Erickson reached out to Badler in 2005 to discuss ways to automate excavation artifact processing. Badler, whose wife is an archaeologist, was familiar with the technical challenge and interested in collaborating. While they admit that other researchers went on to address this particular challenge independently, their initial meeting spurred an ongoing collaboration in a different direction that remains active to this day.

“I’d been thinking about the idea of peopling the past,” says Erickson, who wants to use digital technologies to help put people back into the study of the past. “[Objects] get in the way of the way we write about the past—it’s all about the stuff and less about the people,” he says.

As the advisor for the Digital Media Design (DMD) program, an Engineering BSE degree that combines computer science with fine arts, Badler also understands the importance of getting his students to work with researchers outside of computer science. “Computer science needs the humanities,” says Badler, emphasizing that his collaboration with Erickson has “formed the basis for engaging students to look beyond computer science as programming and instead as a motivator for learning much harder stuff, such as 3D digital representations, right off the bat.”

Pilgrimage to Pachacamac

After working for several years on pre-Columbian Landscapes of Baures in Bolivia, this summer was the first time that Badler and Erickson focused on Pachacamac, a religious site located southeast of Lima, Peru. Pachacamac was first settled in 200 CE and was an important site of religious pilgrimage that drew large crowds from across the region. Archaeologist Max Uhle excavated the site in 1896 and brought a diverse collection of 7,946 objects back to the Penn Museum, including pottery, stone tools, textiles, food remains, baskets, and mummies, all incredibly well-preserved by the arid desert climate.

Before the conquest of the Incas in the 15th century, Pachacamac would have been bustling with crowds and processions. Now, it’s a quiet tourist attraction, devoid of the sounds, smells, noises, and “shock and awe” that one would have felt when entering a place of its significance.

As a starting point toward peopling the site, Badler and Erickson worked with nine research assistants this summer to build a digital Pachacamac. One of the technical challenges is creating realistic-looking human processions using computer graphics models. Typical simulations involve telling a program the number of people and how you want them to move, but with Pachacamac, people need to move and act differently based on where they are—for example, they might need to move slower around a religious site to make an offering. “These are not just pedestrians wandering around,” Badler explains. “They have a motivation, they have some common purpose and direction.”

View large image

This summer, Susan Xie, a junior in the DMD program from Marlboro, New Jersey, worked on ways to parameterize Pachacamac crowd simulations. “For instance, in the marketplace you don’t want them to be dancing—they should be talking to each other or walking around,” she explains. Using reference materials from the site’s original excavation, Xie created digital models of Pachacamac buildings and used Houdini, a 3D animation software, to try out different ways that crowds could move through the site.

To help create clothing for the Pachacamac pilgrims, Felicity Yick, a DMD sophomore from Hong Kong, used digital scans of textiles from the Penn Museum, along with motion capture work and 3D modelling software, to animate walking cycles and movements so the clothing would move naturally during the processions. “You don’t want people to halt and be really stiff,” Yick explains. “You want to animate people standing with subtle movements.”

For more detailed views of key rooms at Pachacamac, Adam Canarick, a DMD junior from Woodbury, New York, recreated the interior and exterior of the oracle room, an important place where pilgrims would leave offerings. His challenge was both filling a room with hundreds of unique, realistic offerings while still creating a scene that matched historical descriptions.

“They were describing it as cave-like, so I wanted to convey that feeling but also make sure that the audience could see the composition,” explains Canarick. After studying objects from the Penn Museum to get the right forms and textures for the offerings, Canarick also “sculpted” the wooden oracle statue and added a skylight and a fire to help illuminate the scene.

View large image

This preliminary work is not only a starting point toward peopling Pachacamac but can also be used to answer archaeological questions. Digital recreations can help researchers visualize different types of structures for buildings where only foundations remain, see how quickly pilgrims could enter and exit a place if there were a limited number of entry points, or how a speaker located at the far end of the site would have been heard by those standing far away. “The archaeological past is partial,” says Erickson, “The digital realm allows you an infinite number of reconstructions to try different things.”

Historical meets digital in the classroom

Students in the Visualizing the Past/Peopling the Past course come from all four of Penn’s undergraduate schools and coursework includes writing about media depictions of ancient people and learning how to build 3D digital models. Students hear about the technical challenges of creating photorealistic depictions of archaeological sites and historical events from Badler and learn about the history of Pachacamac and Penn’s collection from Erickson and curators at the Penn Museum. Students also learn how to use Maya©, an animation and 3D modeling software, to digitally recreate Pachacamac objects for their final project.

Josh Nadel, a DMD senior from Chappaqua, New York, leads students through the process of creating 3D models in Maya. As an introduction to the program, students built a model of a two-handled pot from the Penn Museum from scratch. The process is challenging and Nadel explains that even for professionals, creating seemingly simple objects like this take time. “On a piece of pottery like this, a modeler could do it in a day, then you pass it to somebody else to do texture—roughness, bump variation, cracks, stone grain. That might take a few days,” he says.

View large image

While technologies for “push button” scanning and 3D modelling are available, Badler and Erickson want students to work more directly with the Penn Museum objects to better understand what they were recreating digitally. “The only way you can do that is to touch the object, see what’s connected to what, what are the different materials,” says Erickson. “Even simple objects can be complex, and you have to capture the complexities.”

For their final projects, students created models of artifacts, activities, and architectural features of Pachacamac and the surrounding landscape. Each student discussed the role that their item or location played in Inca culture, what approaches and tools they used to create a 3D model, and what next steps would be needed to add additional realism or context.

This year’s final projects included a terrain model with bridges and stairways along the Inca “royal road,” a 3D model of one of Pachacamac’s main buildings showing water flow within the integrated canal system, and depictions of activities such as making pottery and brewing chicha, a native maize beer. Within each project, students created digital versions of several objects from the Penn Museum, including pottery, plows, hoes, and fishing supplies. Beyond creating the 3D models, students also learned about the abilities and expertise of the Inca culture, added people interacting with objects in scenes to give them more life, and gained a better understanding of how the objects and materials were put together.

One project used mummy bundles and textiles from the Penn Museum collection to digitally recreate a scene from a procession. Andean mummies were not kept permanently in burial sites but were paraded during important religious events. This particular recreation helps viewers not only observe the mummies within a “living scene” but also allows them to see the vibrant colors and patterns of the textiles as they would have appeared hundreds of years ago.

The goal is to use these materials in a future exhibition that tells a social story about Pachacamac—one that sees its history, people, landscape, objects, and even Uhle himself, as connected characters. “It’s a very anthropological way of approaching a museum exhibition,” explains Penn Museum’s Mellon Director for Academic Engagement Anne Tiballi and co-curator of the proposed exhibit alongside Erickson. “And we’re trying to do it from a perspective that’s respectful of indigenous Andean ways of understanding that connection.”

“My goal is to use digital to carry [the collection] to a new level, to be honest about what we don’t know, and get the visitor engaged with how we think about our interpretations,” says Erickson, adding that their proposed exhibit will make use of the provenance of the 130-year old collection, a rare degree of context not always available for objects in a museum.

Interdisciplinary attitude

View large image

The ability to see connections between the humanities and computer science is a powerful tool—one that Myles Al Yafei, a DMD senior from Muscat, Oman, one of the teaching assistants for this fall’s course, hopes to use in her career. Al Yafei spent the summer of 2017 reconstructing an interactive landscape of Baures and through her work was able to see graphics technology and its potential applications in a new light.

Sharing her own experience growing up in Oman, where there are many unmarked historical sites, Al Yafei sees technology as a way to educate and provide insight to help people better connect with their past. “People talk about history as if it’s not related to them, it’s not part of who we are today,” she says. “When you have access to [computer graphics], you realize the similarities and how things repeat themselves.”

Al Yafei, Canarick, Nadel, Xie, and Yick enjoy working on projects that are not just art or engineering and want to become technical directors at animation studios—people who “speak the language” of both engineers and artists. Canarick adds that having a skillset that includes fundamental computer science as well as design, art, and cinematography can also make for more effective storytelling. “It all starts in the humanities, but you need the computer scientists to bring the vision through to fruition,” he says.

Regardless of where students end up working, Visualizing the Past/Peopling the Past demonstrates the importance of being able to use tools and perspectives from the humanities when bringing things to life in a digital setting. “We’re giving students experience using these tools but also an understanding of how those tools are used in a cultural context and in a technological sense,” says Erickson.

And with digital technologies poised to become more advanced and widespread in the coming years, being able to combine cutting-edge computer graphics research with an ability to understand how individuals, cultures, and past events are portrayed will be essential. “A lot of technology in computer science is motivated by problems that are extrinsic to the field,” says Badler, adding that this collaboration “takes the field of computer graphics and gives it a motivator that involves humanistic, cultural, anthropological, and archaeological considerations.”

“That’s where I think the value in our collaboration lies: We can bring students into this environment and challenge them with questions that are not only meaningful but that also have an impact on technology,” says Badler. “The students’ experience learning 3D modeling goes far beyond this course and its application to the Inca culture—realizing the translation of tangible and imaginary worlds into digital models is a powerful addition to their academic toolset.”

Part of the Visualizing the Past/Peopling the Past summer research projects were supported by the Center for Undergraduate Research & Fellowships.

Norman Badler is the Rachleff Family Professor in the department of Computer and Information Science in the School of Engineering and Applied Science at the University of Pennsylvania. He is also the Founder and Director of the Digital Media Design program and Director of the SIG Center for Computer Graphics and the ViDi Center for Digital Visualization.

Clark Erickson is a professor in the Department of Anthropology in the School of Arts and Sciences at the University of Pennsylvania. He is also Curator of the American Section of the Penn Museum.

Homepage photo: Josh Nadel leads students through creating a 3D model of a two-handled vessel. “Understanding the 3D space that you see on your 2D screen is something very conceptual,” says Nadel. “The more practical skills that come next is understanding how to represent a 3D form that you see in an image or in your mind with points, edges, and faces.”