Researchers from the Departments of Computer Science at the University of Ilinois and the University of California, Berkeley have collaborated on an experiment that allows individuals from both locations to see one another and interact in cyberspace in a 3D format. They conducted the first of three experiments, testing the technology, on Feb. 13. Dancers Renata Sheppard from the University and Lisa Wymore from UC Berkeley danced simultaneously, thousands of miles apart, while viewing each other in 3D over advanced network technologies of Internet 2.

TEEVE, which stands for “Tele-Immersive Environment for EVErybody”, is a cross-layer control and streaming framework over a general-purpose delivery infrastructure, and the backbone of the experiment. It is the first system to support extensive experiments across Internet 2 using 12 3D video streams, and achieving between 4-5 3D frames per second across the country. This allows for real-time rendering on the receiver’s side and real-time feedback to the viewer at her request to move in the cyberspace.

The effort of this team of researchers, which is led by Professors Klara Nahrstedt from the University and Ruzena Bajcsy from UC Berkeley, marks the first time that a team has conducted this type of experiment using commercial off-the-shelf (COTS) components over Internet 2 in tele-immersion systems.

Other systems, such as CAVE, Hello Room and Hewlett Packard’s System Coliseum, use very expensive, high-performance equipment that makes them impractical for research at universities. With the TEEVE, the University and UC Berkeley researchers have set out to explore tele-immersion using only equipment that is commercially available to universities at a reasonable cost.

At the first experiment, Sheppard and Wymore danced in three speeds – slow, normal and fast – and tried to remain consistently paced while viewing one another in 3D. The dancers also experimented with forms of interaction. At one point, Sheppard made a motion to fall backward while Wymore reached out to catch her, all of which was projected onto the 3D models.

The TEEVE application can be used not only for performing distributed physical activities such as dancing, learning sports and physical therapy exercises, but also traditional video conferencing.

The team plans to continue exploring the technology through future experiments involving tai-chi instruction, discussion among small groups at both sides and meeting family members in cyberspace.

Through the experiments, the researchers aim to address and solve challenges surrounding real-time 3D video reconstruction and compression, coordination and synchronization of multiple 3D video streams, multi-tier Quality of Service-aware semantic protocols for video streaming and association between 3D content and multi-view displays.