A research project at Microsoft Research Cambridge has brought forth a prototype called Holodesk, which lets you manipulate virtual objects with your hand. You literally “get your hands on” the virtual display. According to the official description from its creators, there is at work a “novel real-time algorithm for representing hands and other physical objects” allowing physically realistic interaction between real and virtual 3-D objects.
Holodesk does its magic with the help of an optical see-through display plus Kinect camera. The illusion is that the user is directly touching and maneuvering 3-D graphics. According to the project notes, a virtual image of a 3-D scene is rendered through a half-slivered mirror and spatially aligned with the real world for the viewer.
A Holodesk video shows the user looking down on a pane of glass at virtual but very realistic-looking balls and other shaped objects. His hands are positioned underneath the glass and they move in such a way that you think the user is actually moving real objects around. He scoops real balls into cups, or so it seems. The video as a whole is a deft rendering of how the virtual and real meet. While Holodesk is not the only 3-D interaction experiment out there, observers say it stands away from the pack, with its use of optical devices called beam-splitters and a graphic processing algorithm, in providing a life-like experience.