Virtual reality in an ancient world

Standing atop the scaffolding in the center of a closed gallery at the Penn Museum, conservator Molly Gleeson spoke facing a small camera on a tall tripod, in a few sentences explaining the yearslong project to select and conserve objects for the next step in the Building Transformation project—the Ancient Egypt and Nubia Galleries.

“I’m really excited to show you some of the behind-the-scenes conservation work we are doing,” Gleeson said. Then she turned and asked “How was that?” to the filmmakers, all Penn students. They were emerging from behind towering stone columns, because the 360-degree virtual reality camera captures the entire space. It was their idea to have Gleeson on the scaffolding.

“I think it’s interesting for the viewer to be able to see the whole gallery from a higher angle that you don't get from just being on the ground level,” says Maya Pratt-Freedman, a fourth-year sociology major and cinema & media studies minor who graduated earlier this month.

The team of four students also filmed in the Museum’s off-site warehouse where monumental Egyptian artifacts are being conserved and readied for the reimagined galleries. “I have to say that it was really hard for all of us to kind of continually remind ourselves that there was no behind-the-camera,” says Gleeson, the Schwartz Senior Project Conservator. “So anywhere you were in the space you were in view. I know a few times we accidentally walked into shots.”

The film starring Museum conservators was one of three made by teams of Penn students in The Virtual Reality Lab course taught by Peter Decherney, professor of English and cinema and media studies in the School of Arts & Sciences. The films focus on the Museum conservators, curators, and senior archivist, telling their stories of engaging with the collection while working in their spaces.

Creation of the three films taught the students and the Museum staff about virtual reality filming while also chronicling this moment in the galleries’ transformation.

“We’re envisioning this as the Egyptian wing for the 21st century. Once this is done, it’s not going to happen again for a long, long time,” says Josef Wegner, curator of the Egyptian section. “It is an important, pivotal moment in the Museum’s history.”

The filming came at a perfect time, he says. “We’ve finalized the vision for the galleries and we’re working very closely with conservation on the treatment and preparation of those materials,” he says, adding that while the Museum staff meticulously documents their work, they don’t document themselves in the process. “I think it’s going to be a wonderful set of videos to have telling that story.”

And the filming has been “a new and very different way to involve students in the Penn Museum,” says Wegner, also professor of Egyptology and Egyptian archeology in the Department of Near Eastern Languages and Civilizations. “It’s really neat to see these students get excited about the objects and the future potential of our Museum, and the richness of the holdings here, and what the Museum might become in the future.”

Virtual reality at Penn

Decherney—an author, photographer and documentary filmmaker—started teaching virtual reality filmmaking in the Virtual Reality Lab course at Penn six years ago. He included 360-degree cameras in a 2018 Penn Abroad summer course that took undergraduates to Kenya to make documentaries with residents of the Kakuma refugee camp and Kalobeyei refugee settlement, in partnership with the nonprofit FilmAid Kenya. The next summer he took students to Puerto Rico to make virtual-reality documentaries about artists and the impact of Hurricane Maria on their lives and work: “The Heart of Puerto Rico” has been shown at film festivals, and won an award for best documentary experience at the 2019 AT&T Film Awards. In the summer of 2022, Decherney returned to Kenya with students to create an open online virtual-reality filmmaking course and establish film clubs in secondary schools.

This semester Decherney launched the Penn Global Documentary Institute, an umbrella for several initiatives involving filming, including the Virtual Reality Lab course.

“It’s a new way of telling stories and creating experiences,” Decherney says about 360-degree cameras. “That’s why I’ve been committed, not just working in Philadelphia, but to taking students and VR cameras to lots of different places globally to partner with communities and document their experiences."

In the course, students “work collaboratively to really have a hands-on experience to learn about the theory, but also the practice,” while creating a useful film for a real-world partner, Decherney says. Each year the class works with a different partner; for example in 2020 it was the Philadelphia Museum of Art.

This spring semester course, funded in part by the SNF Paideia Program, was filmed in the Penn Museum’s Egypt Galleries, the Museum’s off-site warehouse, and the Museum Archives, areas both open and closed to the public.

Some of the 12 students in the class had experience filming and with virtual reality before, but most had not. “We draw from a range of students with different backgrounds: engineers, art historians, and filmmaking students, and students that have never touched a camera before,” Decherney says. “They bring lots of different kinds of expertise.”

Students use 360-degree cameras, and editing software, ambisonic (or 360-degree) microphones, and other equipment provided by the Penn Libraries’ Vitale Media Lab. It was a steep curve learning to operate a camera that shoots very differently than the ones they are used to, Decherney says, but “thinking in VR” can be the most challenging. “Staging, composition, camera placement, are all incredibly important and really do a lot to mediate the experience,” he says.

“You want to find a shot in which you have something in the foreground and something in the background, even though it’s in 360,” he says, “so it’s very a different way of thinking about telling a story and composing a shot.”

Editing the film was also a new challenge, the students say, although for Pratt-Freedman it was her favorite part. “Especially with a documentary, you don’t know what you’re going to get, so much of it is putting together a puzzle in the editing room,” says Pratt-Freedman, who is from San Diego. “I think there’s so much magic that comes out of it. And also with our teamwork, there’s an excitement seeing the shots and reviewing footage as it’s all just becoming more real and coming together.”

Learning behind the scenes with conservators

Before the students started filming, they conducted research, meeting with the Museum teams to discuss their topics, approaches, and possibilities. The conservator filmmaking team was briefed about the warehouse prior to a full day of filming. “I was impressed because they really had a plan, and they had different ideas of what they wanted to film, and we gave them some feedback on how that might work,” says Gleeson.

The Museum team gave an on-camera tour of the warehouse where the ancient monumental architectural elements—stone columns, doorways, windows, and wall paintings—are under conservation. The columns, originally 30 feet high, will be part of the reconstructed ceremonial royal palace of Merenptah in the Museum’s Egypt Gallery, located on the Upper Level. But now, the sections, called column drums, are undergoing painstaking repair and restoration.

The students filmed a time-lapse of the conservators using equipment to lift and flip over one of the column drums, which can weigh up to 12,000 pounds, an experience Pratt-Freedman says was impressive. They also used time-lapse to show conservators applying an adhesive to reconstruct limestone fragments. “It’s really cool to be there and it’s fun to see the behind-the-scenes of it,” Pratt-Freedman says.

Another day Gleeson brought the student filmmakers into the Penn Museum’s closed Egypt Gallery on the Lower Level, where ancient objects that are too big to transport offsite are waiting to move upstairs, including the tallest portions of the columns. “I did end up climbing up onto the scaffolding, which we use when we conserve and install pieces,” Gleeson says about her narration. “The students were interested in capturing the room from above, and I thought that was a really neat idea.”

Although she had experience filming with a 360-degree camera during last summer’s project in Kenya, Pratt-Freedman says it was a new challenge in the Museum spaces. “With a 360 camera there’s nothing you can hide. So it’s about what are we putting in the frame that we want people to focus on. What are we drawing attention to?” she says. “How can we signal that? And kind of exploring the new conventions of 360 storytelling as we go.”

Gleeson says she and her team relished the chance to work directly with Penn students. “It was really great to see their creative process in action,” she says. “I think it’s really difficult for people to understand the nature of our work until they’re physically in this space … virtual reality films could be a really terrific way to give people a sense of being there.”

Chronicling a new vision with curators

The stars of the curator team film were husband-and-wife Josef Wegner and Jennifer Wegner, two Egyptologists who are part of a larger group planning the reinstallation of the Ancient Egypt and Nubia Galleries. They gave an on-camera tour of the Upper Level Egypt Gallery, which will be open to the public throughout the summer, but will close in the fall to make way for construction.

“We got to see some of the artifacts that are still in that gallery, and we also got to see some of the artifacts that they’ll be adding to the new one,” says Shelton Mercer, a cinema & media studies major from West Philadelphia, a fourth-year who graduated earlier this month. “It was really cool to experiment with the 360 camera to capture those different scales and depths.”

The students had “all kinds of interesting questions and insights for us, many of them ones that we hadn’t really thought of, actually, in telling the story” of the galleries, says Josef Wegner.

View large image

The Egyptian collection has nearly 50,000 objects, from pre-history into antiquity, “so there’s a huge potential for really telling the story of Egypt from its early origins and how it evolved over thousands of years,” he says. The reinstallation will include many artifacts that have never been on display, telling a “new cohesive story” with eight thematic galleries linked together.

Penn archaeologists excavated the palace Merenptah from 1915 and 1923 in Memphis, the political capital of ancient Egypt. The Museum’s Egyptian wing was completed in 1926, but the floor for the second-story gallery was not strong enough, so the palace was arranged in pieces in the first-floor Lower Egyptian Hall. The reconstruction will include reinforcing the floor and building a freight elevator so the palace pieces can be moved to the second floor and displayed at full height.

Mercer, a videographer and photographer, has experience with 3D animation. For the film, he worked with the curators to create a 3D environment of the palace room to give viewers a sense of what it will be like when it is completed. He plans to visit the galleries when they reopen. “It’ll kind of be like a hurrah moment because I feel like we had a part in it almost,” he says.

Digging into the Archives

As they have been reimagining the galleries, the conservators and the curators have been poring over documents, photographs, and drawings about the Egyptian collection in the Archives, says Alessandro Pezzati, Museum senior archivist.

For the students’ filming, two conservators—Michaela Paulson and Julia Commander—were looking at materials with Pezzati. “They found photos they hadn’t seen before, and they were really excited. So it actually turned into a real research request,” he says.

The film focuses on 100-year-old diaries and field registers of archaeologist Clarence Fisher, who led the excavation of the palace. “I also pulled a number of the beautiful drawings of the columns and other architectural elements,” Pezzati says, “and photographs of the site showing excavation in progress, some of the columns right where they were found.”

Tianyan “Nora” Han was a third-year on the Archives team. Working with Fisher’s diaries “was like almost having a conversation with him after all these years about the experience when he’s digging onsite in Egypt,” says Han.

Han, who is from Beijing, is majoring in logic information and computation and minoring in cinema & media studies in the College, while also pursuing a major in computer and information science, and an accelerated master’s degree in robotics in the School of Engineering and Applied Science.

“I really want to explore this new form of film production,” says Han, who had not previously filmed a story using a 360-degree camera or edited a virtual reality film. There is synergy between the 360-degree video and the work she does with computer vision, or machine perception, which investigates 3D rendering, she says. “I hope to do more research on computer vision, but at the same time I really wanted to express my creativity.”

A virtual reality future

Pratt-Freedman says she plans to use 360-degree cameras in the future as she heads to a new job as a content developer at Apple. Mercer, too, expects to use the virtual-reality experience he has gained in his filmmaking startup, Merc Media.

And Wegner says he wants to incorporate virtual reality filming into his work, taking the cameras on archaeological digs. “I’m thinking of all the different applications that we could use it for both in field work in Egypt as well as here in the galleries in the Museum,” he says.

The students, faculty, and Museum staff say they hope that the films can be used to help keep visitors connected to the Egyptian collections while galleries are closed during the building transformation, and give then a behind-the-scenes sense of their work.

Museum staff got a first look at the conservator, curator, and archives films at the end of the semester, the students manipulating the 360-degree view as they projected them on a big screen. Others tested watching the films on their phones. And Gleeson tried a fully immersive experience, Decherney explaining how to use the white headset that covered her eyes.

“Oh, wow,” Gleeson said as she watched, “it’s amazing.”

Links to conservators, curators, and Archives student films are on YouTube.