This New Technology Lets You 'Touch' Objects Onscreen—and Could Change CGI Forever

Meet the future of CGI and augmented reality.

Since the 1960s, the best way to simulate an object's motion in space has been 3D modeling. An integral element of the CGI process, it's a cumbersome, expensive endeavor, one that requires hundreds of hours of manpower, state-of-the-art technology, and ever-changing algorithms to produce even the shortest sequences. 

Interactive Dynamic Video, a new technology developed by MIT's Computer Science and Artificial Intelligence Lab (CSAIL), could significantly ameliorate the process. By calibrating the physical behavior of objects—analyzing vibrations in different frequencies in space—IDV can predict how objects will move in new situations, an unprecedented achievement in motion graphics.

It could reduce the cost of the CGI process by eliminating the need for green screens.

Using the technology, you can reach in and "touch" objects in videos, manipulating the environment according to your movements. This functionality has huge implications for virtual reality and augmented reality. An AR game such as Pokemon Go can insert virtual characters into real-world environments, but the two worlds remain distinct. IDV, however, permits interactivity between the virtual and the real, such as the ability for a Pokemon to bounce off the leaves of a real tree, as shown in the above video.

In the near future, IDV will likely prove even more beneficial to the film industry. To create models of virtual objects or environments in the frame, filmmakers today must implement the time-consuming and potentially expensive green screen method. IDV could supersede green screens by enabling a cinematographer to make minor edits in-camera—such as masking, matting, and shading—to achieve a similar effect in a fraction of the cost and time.