Imagine a three-dimensional view, from across the continent or the ocean, of a heart operation — a view with the same kind of clarity and depth of field that being right in the operating room offers. With a setup like that, a doctor could teach a new surgical procedure simultaneously to hundreds or even thousands of others all over the world.

That’s almost the kind of setup that Assistant Professor of Computer Science Kostas Daniilidis has in the General Robotics and Active Sensory Perception (GRASP) Lab. “It’s somewhat like the holodeck on ‘Star Trek,’” he said. His own version is just the basics, but with a little more power and some infrared light it would be the real deal.

For now, it’s a room with an array of video cameras to provide multiple viewpoints and a group of computers to process the digitized images. In a remote location, a viewer sits in front of a screen, wearing polarized glasses like those used for 3-D horror movies. The screen shows what or who is in front of the array of video cameras. Just as if the viewer were peering through a nearby window, when he moves his head, his view of the room pictured on the screen shifts. The flat screen itself is lost behind the dimensional illusion.

Unlike the illusion of a hologram, which is purely optical, the GRASP Lab’s system is digital and occurs in real time, allowing for give-and-take conversation and action. And unlike videoconferencing, it “compares with being physically there,” Daniilidis said.

The original idea for the 3-D view of a remote space, a.k.a. “tele-immersion,” came from Jaron Lanier, the man who gave us virtual reality, and the research began at Brown and the University of North Carolina, two schools known for computer graphics. But they needed a research partner that was tops in computer vision, so they approached Professor Ruzena Bajcsy, then director of the GRASP Lab. When she left for Washington, D.C., Daniilidis took over the project.

Word got around about the new tele-immersion technology, and the phones started ringing, Daniilidis said. Interest came from not just doctors but folklorists, psychologists, marketing experts and paleontologists. The folklorists wanted to show storytelling. The psychologists want to figure out what the mind needs to believe in the reality of what it’s perceiving. The paleontologists want to allow others far away to view bones and other specimens.

The system has been tested on the Internet2, the broad-band version of the Internet for transmitting high volumes of data. When the data passed between Penn and the other participating schools, it caused a temporary spike in Web traffic. “Tele-immersion is the only application which can deploy the full bandwidth of Internet2,”
Daniilidis said.

Anyplace with an Internet2 connection will be able to have a cubicle set up for viewing remote images, Daniilidis said. Remote or small college and small businesses will not be connected. “It’s not viable for everyone’s home. In the next five years, [tele-immersion] cubicles can be installed in … hospitals, big companies and hotels.” He imagines that small companies or professors who want to use such a cubicle could rent one.

The possibilities tease Daniilidis’ mind. He wonders if tele-immersion would give the aura of authenticity to transmitted theater.

“You can introduce everything that needs a full 3-D feeling, like a dance rehearsal,” Daniilidis said. “We want to make something useful for society and for people.”