16

DSLR + Kinect = Add Depth to Your Footage, Take a Peek into the Future

E.M. Taboada

05.2.12 @ 7:36PM Tags : dslr, kinect, RGBD, RGBDtoolkit, volumetricvideo

What do you get when you map footage from your DSLR onto volumetric data from a Kinect?  How about strange, exciting, sweetness!  A very enterprising fellow by the name of James George has developed the RGBDToolkit, a workflow that allows you to marry the two tools with some very intriguing results.  You shoot through a DSLR attached to a Kinect, and after calibrating both devices to a checkerboard, you can start creating some incredible imagery.  But don’t take my word for it, check these videos out:

Here’s an overview of the project:

And some more sample footage taken at a recent workshop:

Now, if you’re the average DSLR shooter, you might think: yeh, that’s cool, but what does it do for me?  Well, beyond creating some very cool experimental imagery by itself, if further refined, this approach could be used for a wide variety of uses.  Imagine having a multi-DSLR/Kinect setup where you could simultaneously shoot an actor from a variety of angles.  Perhaps by synching them you could use that information to place the image of the actor in a 3D CG environment, and be able to create new virtual camera angles.  Or, motion capture style, map alternate skins onto the figures being taped.  Or, perhaps map out a room, and recreate it virtually.  Many of those things are being done already, but for many millions of dollars more, and here we have two off-the-shelf tools along with free open-source software that let us start dipping our toes into that world!

Who knows?  Perhaps volumetric scanning will become a common feature on future cameras, allowing for 3D projections, holograms and other assorted weirdness.

Just to be clear, I don’t know if you can do the things I just suggested without some major work, what I’m saying is this is an interesting step in that direction (maybe some of you more tech minded are already thinking about how you could pull it off).  For those who just want to try this out right now, as is, things might look a tad “crude” if you’re looking for a photorealistic representation, but if you are willing to explore a stylized look (think Tron or a A Scanner Darkly) you could create some very cool stuff.  You might even decide to go for extreme stylization by perhaps creating some script/filter that dynamically messes with the dimension and color channels (i.e separate an image into layers based on colors).  I can see myself spending many an afternoon playing around with this.

Ready to try it out?  You can download the software along with instructions for a DSLR/Kinect mount at the main RGBD Toolkit site.  For more sample footage, check out their Tumblr page.

If you get a chance to play with this, let us know what you think in the comments below!  Can you see other uses for this?

[**Correction**: Attribution error - James George is an artist fellow at the STUDIO for Creative Inquiry, and received funding support from the Playmodes audiovisual collective.  If you're in the NYC area, go check out an upcoming exhibition that shows off a 17-minute documentary made using the toolkit!]

[via The Verge]

Related Posts

1. What Could the Xbox Kinect Mean for the Future of Storytelling?
2. 'DSLR Filmmaker Toolkit' Vies to Be the Swiss Army Knife of On-set iPhone Apps
3. How to Remove Horizontal Banding Artifacts from DSLR Footage